Problems With Online Ratings Sites | AVVO, Superlawyers, US News
What Are Problems With Online Rating Sites?
The Rise in Legal Search on the Internet.
Examples of attorney rating sites are Martindale, Lawyers.Com, and Superlawyers. At least 78.1 million people in the United States use the World Wide Web. So, it stands to reason that these people are likely to use the Internet to locate a personal injury attorney.
Out of 228 million adult users of the Internet, 93% browse local businesses and other service types. This number includes law services. Such a large amount of traffic has caused an increase in questions. A small industry has taken over legal marketing on the web through attorney-ranking sites.
What Are Attorney Rankings Sites?
The popularity of attorney ranking sites has continued to increase over the last several years. Their purpose is to provide people with searching for legal services and professionals. Consumers are hoping for a resource they can trust.
These sites generally list attorneys that have the highest qualifications regarding:
- Educational background
- Win/loss record
- Client satisfaction
- Various other factors.
Profit is what drives attorney ranking sites. Some of them employ ghostwriters to put up a copy on other sites in their blog networks.
Where Does the Problem Lie?
Three attorney-ranking sites are the most prominent. Conflicts of interest abound on these sites. And the approach used is very questionable. Some sites allow you to ad bid on your competitor’s name! (So your picture and NAP Data show up when someone searches for them.) The validity of the rankings generated on all three sites was debatable. The same is also true for its general operating practices.
The Problem With Personal Injury Lawyer Rating Sites – Avvo.
The Story of AVVO
Attorney Michael P. Ehline of Ehline Law Firm Personal Injury Attorneys, APLC, is one of the first 5000 or so original Avvo members. His partial public profile is above for reference. Michael Britton Avvo founded AVVO in 2006. Formerly, he was an expedia.com executive, according to the AVVO website.
He proclaimed that there must be a more efficient way to choose experts besides ‘pay-to-play’ Yellow Pages. At that time, attorneys and physicians frequently used the Yellow Pages to advertise their specialized services. Here is an example of an attorney user profile on AVVO. The mission of Avvo is to ensure people have the best opportunity to make sound decisions for vital matters like legal and medical requirements.
How is the AVVO Rating Achieved?
1. AVVO lets consumers look at professional information about attorneys. So it includes reviews by clients about specific attorneys. But it also contains peer ratings by other lawyers. 2. Avvo has its lawyer ratings for no cost on its website.
An Examination of Avvo’s Criteria and Practices.
The process is as follows:
1. With scrapers and other data, AVVO creates profiles of at least 90% of all practicing attorneys. And this includes professional information about the attorney and a rating by AVVO. 2. An attorney can claim the free profile Avvo creates, and they can edit it as many times as they’d like via the use of their username and password assigned after registration. 3. Once an attorney claims their profile, the choices below are provided: a. Keep the basic profile free (contains ads purchased by other attorneys). b. Upgrade your profile to “Avvo Pro” for a monthly fee of $49.95. (does not have ads and provides advanced marketing tools). c. Purchased advertisements: d. An attorney’s ‘Sponsored Listing’ includes a picture the attorney offers, preferred tagline, and the choice of having their rating and customer reviews by Avvo on their listing. Also, AVVO randomly places sponsored, paid ads on its advice and search sections.
These are for consumers to use when they are looking for law services.
e. Display advertisements f. The attorney is paid for a display ad and is placed on the pages for attorney search results, competitor profiles, Q & pages, and legal guides. (pages for legal advice). Of course, potential clients typically see the paid ads first.
The more time an attorney stays on Avvo’s site, the more business potential clients they are likely to generate. Ironically, Avvo’s site is somewhat like the advertising model ‘pay-to-play’ version of the Yellow Pages. Ironically, that purportedly caused Britton to start the AVVO website, to begin with.
AVVO assigns a calculated rating to each lawyer’s profile. AVVO’s opinion on the importance of attorneys’ profile information is what these weights reflect. But legal expertise ratings are just the general opinions of AVVO.
Ultimately, the more innovative and technical the firm or their SEO guy, the better they rank on AVVO. Similar to Google, AVVO also uses a mathematical algorithm. So it creates ratings from the use of its secret numerical system.
The basis used for the Avvo rating is background info. But the weight assigned to the information remains a secret. And this is mainly because Avvo does not want bets made on the rating system. AVVO states they want reliability. So this is why their inner workings are secret. But reverse engineering can tell you a lot about the AVVO algo.
In Summary
Avvo’s attorney ratings come from AVVO’S proprietary secret system. And various kinds of attorney information get assigned specific weights.
The basis of Avvo’s ratings is as follows:
- Critical factors in selecting an attorney, according to Avvo’s opinion.
- AVVO delivers the data to the AVVO attorney profile when provided with attorney information.
- How the information gets weighted remains secret.
- Regardless of the weight, no proof exists that attorney information is factual.
- Information obtained from the Internet is not verified, not a precise source for getting a true story. (online user reviews and even colleague ratings can be faked using Algo-defeating methods, voting rings, etc.)
- Avvo has demonstrated many cases of researching lawyers’ experiences where the data gathered misrepresented the attorneys’ information. For example, the specialties provided by attorneys have not been correct.
- Attorneys are not assessed, interviewed, or met in person when information is collected. The attorneys are being evaluated blindly.
The following describes Avvo’s practices:
The mission of Avvo is to provide consumers with the help needed to make sound choices when they require legal or medical services. A necessary amount of information about attorneys in their general area is provided to consumers. Of particular interest, the search slows down due to paid ads promoting competing lawyers.
Our Closing Opinion:
Avvo is an intelligent business model for those wishing to make money off publicly available attorney information in a world of automated algorithms. And they charge a fee to enhance the listings or sell other forms of advertising to lawyers and law offices. For finding an attorney, it is probably better to go online and do a search.
That way, you browse the recent verdicts and settlements of the website you like. Then you can call an attorney who looks good.
All online user ratings are suspect. Semi-intelligent people can easily fake or game AVVO. So AVVO should not be the last word in your search for the best personal injury lawyer.
AVVO is helpful if you pay to get your listing on the home page, for example. Other lawyers and consumers may click your enhanced or PRO listing and drive traffic to your website. So at least this can help your Alexa rating, etc.!
Understanding US News and World Report Lawyer/Doctor Ranks
The United States News and World Report’s abbreviation is “USNWR.” For many, it remains a trusted and well-respected resource. For example, you can find an attorney, doctor, and college rankings. Since the late 1980s, this media outlet has released all types of ranking lists.
There seems to be virtually no way to determine these rankings’ persuasiveness. No GOOGLE results exist online when searching the below quotations.
- “United States News and World Report ranking impact.”
- “United States news and world report impact.”
- “Who reads United States news and world reports.”
- “United States News ranking impact.”
If you fail to use quotations in the search, it will only result in USNWR news related to colleges that have lost or jumped rankings in USNWR. By itself, this is an indication of the amount of influence the business has on Google. Also, when researching the ranking impact of USNWR, the specific venture of ranking becomes your focus.
College Rankings
The USNWR college rankings appear when someone tries to locate potential colleges online. But the criterion used by USNWR to decide its rankings is questionable. USNWR claims that objective data was used to populate their rankings. But was flawed and subjective information used?
The so-called investigators seem to look at the college rankings most often. In 2008, the American Board Association Journal released a Journal which revealed flaws in some of the USNWR criteria used to determine colleges’ ranks.
The results of the student’s potential from law schools then get reviewed. So this is what the selectivity becomes based upon. Undergraduate GPAs, students’ median LSAT scores, and law school acceptance rates are the basis for 20% of the USNWR ranking. And the Association of American Law Schools found that many law schools advised students to submit their applications. And this happened even though the schools knew these applicants were unqualified.
Law schools instigated this process to bump up their acceptance rates, which would, in turn, bump up their rank in USNWR. So this seems to be another hint of the influence USNWR holds over people. In addition, though the GPA of undergraduates appears to be objective, they are subjective.
There is a difference in the difficulty level, grading scale, and required materials. The difficulty level and the classes remain different. And this remains true even though the requirements for course criteria are identical. So the apparent effect is the ranking process of USNWR becoming based upon the student’s GPA.
Criteria Examination
So, when it comes to ranking lawyers, how does it work? In September 2010, USNWR and Best Attorneys presented their best lawyers’ selections to the media. According to Best Lawyers’ website, the most respected and oldest publication of the legal profession’s peer-reviews is “Best Lawyers.”
Many people think being listed on Best Lawyers is a big deal. But ‘The Best Lawyers in America,’ ‘Best Lawyers,’ or ‘The Best Lawyers in Canada,’ don’t want you to call yourself the ‘best lawyers.’ And listed lawyers can’t even suggest that a non-listed lawyer is not as good as one on their site.
USNWR Ranking Attorneys
Like USNWR colleges, a similar process exists to rank attorneys on Best Lawyers. The rankings methodology behind the joint venture involved surveying clients of numerous law firms. They also look at marketing, associates, and recruiting officers. And they also study the managers and attorneys of leading law firms. So they asked each person what they thought was vital for clients. Plus, they researched the actual attorneys looking for employment at law firms. Next, they looked at selecting an appropriate firm to refer to legal work.
According to USNWR, questions on the survey covered reputation information. Allegedly there were six studies to determine the criteria.
Below are the six different surveys. And related questions EXIST under the ‘methodology’ section.
- Fifty-two thousand four hundred eighty clients received client surveys.
- What was the process used in client selection?
- Were any of these so-called selected clients practicing law?
- Forty-three thousand nine hundred attorneys received attorney surveys, including all the lawyers listed in Best Lawyers in the United States.
- What was the process of selecting attorneys not listed on the site Best Lawyers?
- Who benefits from being on the Best Lawyers site? How does this not pose a conflict of interest?
- Two thousand three hundred fourteen marketing officers and 2,322 recruiting officers received a survey.
- Are the listed attorneys related to board members, recruiting, or marketing officers (nepotism?)
- So we know law firms hire each of these officers. How does this not pose a conflict of interest?
- Why weren’t these surveys categorized?
- Eight thousand five hundred ninety-seven firms without contact with recruiting or marketing offices received surveys.
- Were paid advertisers used to assist in the so-called study?
- Why weren’t these surveys categorized?
- One thousand seven hundred seventy-five summer associates and 2,322 firm associates received surveys.
- Again, how does this not pose a contact of interest?
They Posted Five survey types on their site. It brings to question if there was a 6th survey. If so, what kind, who received it, and what information did it hold? In Summary
The process for deciding the rank of law firms’ was as follows:
- Ask “prominent” (whoever this might suggest) citizens in the law community what they believe the acceptable lawyer’s criteria should be.
- Based on the responses received, generate and send surveys specific to certain law community groups. Not reported is the apparent conflict of interest the surveyed people could have. There is also no proof that the people surveyed didn’t help create the survey itself.
So use a combination of the following to rank attorneys:
a. Data from completed surveys. b. Information that Best Lawyers provided iii. As stated by the USNWR, Best Law Firms, on top of data gathered from the studies, evaluates around 39,372 well-known attorneys. Most of all, subjective and quantitative factors are the basis for USNWR rankings.
For example, a factor some people consider might be ethics.
What is disputable?
- The process USNWR uses to decide who will get a preliminary and secondary survey.
- Also, the opinion of these people. What makes it more relevant than people not surveyed?
- Why are raters with conflicting interests possibly better than those receiving a survey?
- How does using advertisers and those with a financial interest in USNWR rate fellow attorneys not pose a conflict of interest?
Understanding “Super Lawyers”
The Story
There are a few sites that rate purely personal injury lawyers. But there is one that breaks down the award by type of law practiced. Super Lawyers (SL) is the name of a famous attorney rating service. SL remains different from Avvo in several ways. Super Lawyers do not assign a numerical rating. SL also maintains a printed magazine version. That publication was around before they even had a website.
Many attorneys consider Super Lawyers to be an absolute honor and achievement. It is somewhat of a peer review system. Contrast this with Avvo. AVVO is both a consumer and a peer review system. But SL remains highly recognized in the law field. And this is especially the case if you can afford a full-page ad in the print magazine. Above is a screenshot of the Ehline Law Firm Super Lawyers online profile.
Rising Stars?
A nomination for being ‘Rising Stars’ may also be in line for Super Lawyers, a much more selective choice. As stated above, not only does Super Lawyers provide a rating service. SL publishes state and region-based paperback magazines. And the content featured in the magazine includes paid ads. But it also offers a list of lawyers and their profiles. Again, you can buy an enhanced user profile on the web version of SL.
ABA law schools and lawyers often receive copies of these publications. Their list of publications includes the rosters for ‘Rising Stars.’ That list features the top 2.5% of attorneys for each state. And the listings for Super Lawyers feature the top 5% of attorneys for each state, based on Super Lawyers criteria.
Process and Criteria Examination?
Super Lawyers generates lists from the four below steps:
I. Collecting The Pool Of Candidates
Three ways exist for an attorney to become included in the pool of candidates.
- Formal Nominations – Attorneys can vote for other observed attorneys in the same state. ‘Safeguards’ exist, so the system cannot be ‘gamed’ by attorneys. Allegedly less weight goes to nominating an attorney from a lawyer’s law firm than selecting an attorney outside their law firm.
However, the process ultimately gives attorneys the ability to nominate attorneys from their firm. Plus, they can appoint family members and friends. According to an article published in a Boston Business Journal, some attorneys feel it is a popularity contest. It is unlikely that Super Lawyers truly recognize a ‘back scratch’ process for nominations. It appears they conduct a tremendous amount of research into attorneys’ personal life and background.
Super Lawyers prohibit candidates from campaigning for admissions. But who enforces the rules?
2. Star Search – Over 150 sources of data exist. For example, databases, the internet, and local and national publications for legal trades. These all help to categorize attorneys possessing excellent credentials. But the sources they use online are not specified. United States News and World Report’s Best Law Firms, AVVO, or possibly even both, are used during selection.
So these secret methods remain questionable. A conflict of interest could occur relatively quickly among employed lawyers researching candidates. For example, they may ‘locate’ a family member, a close friend, or a co-worker from their firm while looking for candidates.
3. Informal Nominations – There is no weight given to open nominations during the rating process. However, clients, readers, and lawyers holding licenses in different states can nominate an attorney. The candidate pool is assessed further by Super Lawyers. Presumptively, ‘readers’ do not refer to other lawyers or peers. But until we know for sure, it would be a conflict for Super Lawyers to list participants of open nominations and not reveal ‘readers’ or how they qualify to nominate.
I. Assessing the Pool of Candidates
Here are the pooled rating factors:
- Settlements and verdicts.
- Transactions.
- Amounts awarded on cases won or settled. This award could be an indication of the quality of a lawyer. But this may not necessarily be true. Sure, a great lawyer may receive awards for getting clients’ money. But a horrible lawyer may only receive award amounts of 5 million dollars, for example, worth 100 million dollars.
Clients represented
- Past clients are assigned a rating weight. Knowing a lawyer’s clients can help determine how reasonable the attorney is. It can also be a tool to help determine the better a lawyer is and the cost and the demand of a particular attorney. However, it is a valid question as to the amount of weight given to this indicator.
- Position held within the law firm
- Experience
- Community service and pro bono service as an attorney
- Awards and honors
- Employment and education background
- Certifications and special licenses
- Writings and scholarly lectures
- Bar activities
- Other professional activity
- Other outstanding achievements
- But this remains a mystery.
Like the criteria used by Avvo, no revelation of the weight given to these indicators publicly exists.
I. Evaluation of Peers
Out of all attorneys in a specific practice area, the pool candidates have the maximum number of points. A ‘blue-ribbon panel’ then rates the lawyers. Attorneys compete to be ‘Super Lawyers’ and also determine candidate admissions. So it looks like a conflict of interest exists. For instance, it would be simple for a blue ribbon panelist to give attorneys high ratings at their law firm.
II. Selection Process and Publishing
Lastly, the size of the firms where the poolees work separates them into categories. Candidates with the most points are selected as Super Lawyers.
Super Lawyers:
- Performs searches on the Internet to ensure no matters exist that would negatively affect the attorney.
- SL checks each pool candidate’s standing against the local licensing authority records.
- Contact attorneys for confirmation that the information published is correct.
- Asks pool candidates to prove that they have never been a part of criminal or disciplinary proceedings.
The elite ‘Rising Star’ only lists 2.5 attorneys from each state. For an attorney to qualify for Rising Star, they have to either be 40 years old or have been in law practice for no more than ten years. Super Lawyer and Rising Stars use the same selection process. But in this case, there is no Peer Evaluation. Also, attorneys pay for featured ads. And Super Lawyers’ paid ads also appear in their magazine publication. As an aside, SL offers five different ad options.
- Standard Profile
- Platinum Profile
- Impact Listings
- Display Advertising
- Online Profiles.
The content and price of each of these options vary. According to Super Lawyers, advertising is not a basis for choosing attorneys to be a Super Lawyer. But attorneys who advertise remain more visible within the legal community.
Because the terms Best Lawyers and Super Lawyers could be a deceptive advertising tactic, New York State considered revising advertising laws in 2006. The publisher of Super Lawyers disagrees, of course. But the SUPER LAWYER awardee has better qualifications to win clients’ cases.
In Summary
Regarding the Process and Criteria used by Super Lawyers
The problems they have are similar to Avvo’s:
- The opinion of Super Lawyers on what they deem a reasonable attorney is a basis for the criteria used for Super Lawyers.
- The weight given to the criteria is not public.
- Non-Super Lawyers and Super Lawyers can pay for a featured ad in each publication.
- According to Super Lawyers’ website, attorneys and law schools receive publications.
- The criteria used are possibly flawed.
- Super Lawyers appear to ignore possible conflicts of interest.
What Are The Results Of The Investigation Of These Sites?
An investigation was decided in August 2011 by ABACE, the American Bar Association Commission on Ethics 20/20. In February 2010, the House of Delegates proposed investigations into the United States News & World Report. Specifically, they wanted to know more about yearly lists of rankings of the top law firms. And this was known as “Resolution 10A.” Also, The New York State Bar Association sponsored this resolution.
Unspecified attorney ranking sites received ABACE investigations. Interviews by disciplinary agencies, attorneys, consumer groups, bar associations, and other ranking sites took place. But ABACE found no evidence existed that pointed to a widespread problem. So most people figure that attorney ranking sites are genuine. However, there is not nearly enough evidence for this to be definite.
ABACE also stated no need exists for the ABA to study, support, undertake, or fund additional resources. One reason for this decision was there was not an adequate amount of evidence that a pervasive issue would merit the ABA to proceed further. ABACE concluded that the expense of employing experts would not be justified if a sufficient amount of evidence did not exist.
