Sex workers are not who you think they are. First of all, sex workers are not all stereotypically or conventionally attractive- in fact I've worked with quite a few very successful women in Baltimore with more than a few extra pounds (The beauty standards of the US Mid-Atlantic region really are far more forgiving of large ladies. Just ask John Waters.). I was one of many women who refused to shave their pubic areas. The first sex worker I ever met had multiple facial piercings and bright red two-inch-long armpit hair.
Second, sex workers are not all desperately unhappy, and many of the ones who are certainly don't seem so at first. You aren't going to know who is a sex worker by looking for the first sad lady with a tale of childhood molestation and an inability to meet your eye. A lot of people have that, anyway.
Third, sex workers aren't all young. I worked with many who were over forty.
Fourth, sex workers aren't "trashy." Plenty get away with telling even close friends and lovers that they are some sort of office worker or consultant for years on end without anybody questioning it because they are composed, articulate, well-dressed women with law degrees and business suits.
Finally, as I just implied, sex workers aren't all "out." This is important. You have, by virtue of existing in our society, most likely met a sex worker who you did not know was a sex worker. You have probably met a person engaged in prostitution. Think about it. How many of your friends and acquaintances do you follow to work every day? How many of your coworkers do you follow home? Why are you so sure that they "aren't like that?"
Sex workers, from strippers to dommes to pornstars to prostitutes, can and do date, and there are quite a few who either don't tell their partners or have no trouble at all finding partners willing to date a sex worker. A beautiful woman who for some reason has no partner and works late nights might just be telling the truth when she says she's a waitress and she's not very interested in sex. You can't tell.
As with trying to figure out who sees sex workers, there is simply no way of knowing. The business is big. Much, much bigger than most people know, and most people involved are understandably embarrassed to admit what they are doing by a society that tells us sex workers are all bad people.