Human-augmented algorithms

Robin · 14 Feb

It has struck me when browsing and reading people's Upwork profiles, that the algorithm-only approach of judging the quality / expertise / professionalism on display in a profile, does not really suffice to surface the best professionals (by which I mean me, obviously).

The question that struck me (even in a world of AI-everything), is why is there no human judgment of what a good profile is (aside from the occasions where there is this, i.e. Expert Vetted).

I realise that Upwork can't read and assess all profiles. But they can do that for Top-Rated Plus and maybe even Top-Rated, too. And maybe even it's a service you could chuck connects at and pay for (pay for an expert review of your profile & a professional manual grading – e.g. after making significant and careful updates to various profile areas).

I know the algo is designed to surface new freelancers too, but I'm seeing some weird results in Talent Search which are occurring consistently and are seemingly not the result of rotation (one profile with two lines of text & no portfolio, certs or anything else (whilst some job history) consistently ranks as the no.1 pick in my country within my field).

It just strikes me there needs to be a human element. Or, AI could certainly achieve some kind of 'assessment / judgment' profile grading on a schedule, on profile changes, or upon request; with a human vetting available for top-level freelancers or also upon request.

This could be something which is quite heavily weighted in the algo.

I know I'm proverbially spilling these words into the wind on here... but this algo feels old and not really all that fit for purpose, and doesn't seem to be surfacing some really good talent, judging from posts on the old board and on this forum, too.

Would like to see Upwork innovating in the Talent Search / algo area with some positive & meaningful intent. There could even be opportunities for them to sell more Connects (their real & actual business)!

RRavi · 14 Feb

Robin
Your post is way too long.

Of course, Upwork uses human input/control.

AI is just a tool.

The problem lies in the human(s) involved.

Robin · 14 Feb

Ravi Your post is way too long.

You're not in a headlock to read it 😉

RRavi · 14 Feb

Robin You're not in a headlock to read it 😉

Haha!

Mark · 14 Feb

I am not sure I understand all of what you are saying but I will say in no way do I want Upwork judging profiles on their "goodness" human, AI, or otherwise.

For me, an Upwork profile is a simple act of marketing. You want client's to read it and see the solution to their problem in human form. Most people treat it like a resume or as a way to present a matrix of "hard" skills that they have. They list a number of years doing whatever, with bullet points and such. An algorithm or a human could readily see that more skills is better, more years is better, bigger words used is better. But a client, who may not even know what any of those skills are might be completely lost.

So, what point is a profile being assessed and found "good" if it is bad marketing.

Alexandra · 15 Feb

I think it would take a lot of (human) resources to verify every detail of any 'good' or perfectly marketable profile. Just imagine having to check every academic qualification. A while ago, I thought that UW was actually planning to do that when it announced the Preferred Freelancer (or whatever it was called) programme costing 50 non-refundable connects. But frankly, I don't care if I don't appear in every search because I much prefer to bid on projects.

Eve · 15 Feb

No matter how faulty Upwork's algorithms are, I trust them more than I trust their staff. At least regarding this.

Kelly_E · 15 Feb

I understand the spirit of the idea. Give newbie clients, some darned measure of quality in the firehose of possibilities they're being swamped with.*

But I'm jumping on Team No, (1) because as Alexandra says, it would take a lot of resources, and (2) because it would take a lot of SPECIALIZED resources. There is no way there are enough staff left at Upwork to know what's amazing about my brand-specialist profile and to know what's awesome about @Eve's translator profile and to know what's top-notch about coders, and voice talent, and legal assistants—if you aren't in those fields, or at least adjacent, you don't know how to judge them!

Upwork would be applying one concept of what's "best" to hundreds of fields, and they would get So. Much. Wrong.

(Even people who want to hire a legal assistant, for instance, may want one for different reasons—so "the best" can be different to different clients.)

*Now—the swamp is a big problem that UW is equipped to deal with... I know this because they used to, back when I was allowed in! Poor quality is easier to judge. And if UW wanted to turn the spigot down on new fls, or cull poor quality fls already on site... that I could get behind.

IMO Upwork is having a danged hard time doing what they are doing. I'd rather they do what they do A LOT better, than try to add in a thing that's the job of the prospective client.

Eve · 15 Feb

Kelly_E Poor quality is easier to judge.

100%

Robin · 15 Feb

A simple thing I am suggesting here is a Bronze, Silver & Gold grading for profile completeness & quality.

This would be a factor driving profile appearance in the Talent Search algorithm, and would be a 'badge' only visible to Freelancers & algorithms (the one which drives the client's view of Proposals relating to Jobs, too).

This would give an indication of how we're placing in those two algorithms, which in my view is currently too opaque to freelancers, and which doesn't really help us to do any better.

The reviews I speak of would simply be a request to have your profile regraded after a re-write, or adding certifications & portfolio items etc. This may trigger a higher profile grading, then leading to a better ranking, which if implemented in this manner would lead to more invites & proposal views.

This does not require any knowledge or opinion on any particular profession on Upwork's part.

It would be about how profiles are put together, and about driving the algos, and about us as freelancers having some knowledge of where we are positioning in those key areas of our discovery by clients on the platform.