The LSI myth is all grown up!

Here I go again… and seriously, I just can’t fucking help me’self. Ya know? I’ve spent many years defending the flame against those dumb ass people pretending to be SEOs. In particular, the ever popular ‘Dave Bait‘ that is Latent Semantic Indexing/Analysis. Yup, even recent pounded it here.

So you can imagine my emotions when news started to come outta the land of Moz about a LDA tool. At first some of the geekier instincts thought, this might be neat. That soon drifted back to fucking reality as the age old words of isolated anomalies took hold. Then of course if hit me like a link spammer on an abandoned blog;

Here we go again. Time for more snake oil and mass confusion!

Leaving the land behind

Now, I don’t want this to become a debate over LDA and what it means. Nor turn into Moz-illa just yet. We need the larger picture first. As with LSA before it, this will become an unholy comedy show for those reading the banter in the SEO space…

Loved these…

“I’m not even going to try to explain the intricacies of LDA suffice to say that it involves the long standing SEO theory of topic modelling which isn’t so much a theory but more common sense. Learn more about the flavour of the month over at SEO Moz ” – Hamlin.com

Uh… wow. Nice one dood. It’s a fucking bril’ idea that you didn’t get into the intricacies my good man. Because that ‘SEO theory‘ bit? It’s a computer science thingy actually. Oh but it isn’t a theory tho? So a ‘common sense’ thingy-ma-bob? Perfect. That just makes me all warm in my guts bro.

Oh and maybe this ‘Practical Summary‘ has some goodies? Awwww for fucks sakes!! Can I please get that 3 minutes of my life back? (U will pay Mikey!). I gotta love the advice…

1. Make sure you are using the rel=canonical tag.

…shit…

2. Make your web site about something.

…. fucking Thomas Edison lives

3. Stop feeding the social media machine all your stuff.

…OK, this is just getting weird.

Oh and please do go and read the thinking here. But don’t bother coming to me to get those precious moments of yer life back m’kay? Then there was some wonderous bullshit floated with sentences such as;

“You can’t get away with two sentence pages and minimally valuable content any more – you have to do the hard work of creating good stuff in order to leverage this algorithm effectively. T ”

So the algorithm likes ‘good stuff‘? And two sentence pages???? Did you miss the whole Demand Media / Mahalo fiasco? meh.

Then there was twitter, which I kept finding gems such as;

We can clearly start to see the evolution of ‘Google’ and ‘LSI’ in some unholy darker shade of SEO hell that it will soon become. Shit, shit shit… Tommy? Go fetch Daddy’s boomerang.

Now, I will leave these fine, if not somewhat misguided, folks alone. Because even that is not the point here. It has already started. And if this fire is allowed to burn out of control, we shall all be consumed by it. Staffers and contractors asking. Clients and department heads asking. Snake Oil SEOs out stealing $$$ under the ‘Google LDA’ banner. We simply must implore people to look at these things reasonably.

Back to the Moz

Ok, now, I will have to lay this back at their door ultimately. Why? Because this is the genesis of the new threat. If they like it or not. For the record I have already written about the whole ‘Google and LDA‘ thang, albeit from a somewhat different angle. But all along I tried to impart one simple concept (pun intended); let’s not sell it for what it isn’t. Also Rand, Ben et al, did seem to try and put that out there. But it isn’t helping.

Some comments on the post, such as these, needed to be answered, to stop the madness (which they weren’t) .. so allow me;

Whoah back there… ‘behind the Google curtain’. Please get up from your desk and go sit in the corner ok? That titanic shit? I have no clue what yer saying. Go to the corner.

LDA in detail? That like a 3 day SEO course? And a link to Bill’s work on named entities? Those are actually some relatively different elements there and strangely, one is ‘semantic’ analysis and the other… well… fucking named entities. And we can’t prove anything. Ok?

What the fuck? Is that English ol chap? Please don’t be all gushy like that. How do you know it is a goldmine until you try it? We are interested in having you tell us if it serves a purpose. Thanks.. I hope you successfully have that dictionary removed from yer …

Not a stretch to assume…LDA or some other semantic analysis? Uhm sure. It’s a search engine, that’s a safe bet. But once again, we’re not deconstructing Google no matter HOW many times we may have referenced them in the post. And that other stuff? Strange but it sure sounds a lot like phrase based IR, not LDA. We haven’t looked at that yet.

Thanks for that. And your welcome for that… and well.. What? Once more we have no idea why everyone keeps trying to imply this is about Google. I mean this page only scored like a 56% for those two terms.

You get the idea. I have never heard so much greek-geek speaking bullshit in all my life. I mean I am reading comments by people that sure sound goddamned smart, but when you read it a third time, it becomes apparent they don’t even ‘get’ what LDA does nor barely semantic analysis at all (synonyms being on of the common foibles). Please Rand/Ben, please do try to hit such comments and discourage the perceptions of Google gold and continue the education into CS/IR..

Help stop the madness

And so my friends, I leave it to you now. Tell peeps to read up on where LDA came from. Get them to learn more about the IR world. Won’t you just help stop the bleeding? Before it’s too late? If you see some LDA bullshit out there, please contact your local SEOBS reporter. Thank you.

I really don’t care who makes an LDA tool. Shit, you can go and grab the code for yourself if you wanted to. It is just super fucking important that people understand that we can’t really get a strong indication of what any search engine is doing with limited data sets as far as methodologies and scoring. It’s a fact, get over it. And if you’re a manic Mozer, please watch the person next to you and ensure they don’t drink too much of the kool aid. Be smart about it and hopefully we won’t all be cleaning up LDA Snake Oil and mass confusion in the years to come.

/As you were…

Peer into Google’s predictive algorithm with Google Sets.

Looking for creative ideas on what to write for a given topic?  Use Google Sets and get some *great* ideas.

Can you make a piece of content that uses all the words in a set but still reads perfectly?

How well do you think this content would rank?  I’ll tell you.

F**cking amazingly well.

I use some made-up terminology in this piece that you should know:

1.  SERPrint:   Your SERP footprint.  I represent this by displaying your numeric position on each search engine in this order:  Google, Yahoo, Bing.  Example: “1,25,0″ would indicate page 1, position 1 for Google, Page 3, position 5 for Yahoo (#25), and no ranking for The Bingers.  A 1,1,1 would indicate top SERP on all engines.

2.  Search Engine Weight (SEw):  The weight of the search engine corresponding to the usage percentage for all searches as reported by comScore March 2010 U.S. Core Search Rankings.   This is the first piece of the formula.

3. Page Weight (Pw):  Using the AOL SERP CTR Study metrics of “90% Page 1, 10% Page 2″ we can conclude that the pattern repeats past Page 2 into deeper pages.

The Factors

There are 3 factors to consider for a cursory SERP score in the context of the search engine’s given results.

1.  Search Engine Weight:  More volume = more weight.

2.  SERP Position:   From 1-10, where does your listing show?

3.  Page Weight:  How many pages within the search results does your indexed listing appear?

Search Engine Weight (SEw)

Using comScore data, we assign the weight of each search engine according to its search volume.

Google:  65.4
Yahoo:  16.8
Bing:  11.5

Observing the Obvious Highlight Moment:  6.3% is spread across the other engines.

SERP Position / Weight (SERPw)

For a given page, how far down in the list is your SERP. Assign weight based on AOL SERP Score data.

AOL SERP CTR Study

We need a big sample to determine the CTR on a given page for a SERP. We’ll be using AOL’s SERP CTR study to get the dispersement %.

Total Searches: 9,038,794
Total Clicks: 4,926,623

% of clicks

• • Click Rank1: 2,075,765 42.13%
  • Click Rank2: 586,100 11.90%
  • Click Rank3: 418,643 8.50%
  • Click Rank4: 298,532 6.06%
  • Click Rank5: 242,169 4.92%
  • Click Rank6: 199,541 4.05%
  • Click Rank7: 168,080 3.41%
  • Click Rank8: 148,489 3.01%
  • Click Rank9: 140,356 2.85%
  • Click Rank10: 147,551 2.99%

Page Weight  (Pw)

How deep within the results does your SERP appear? We use the AOL SERP CTR Study to conclude that it’s 90/10 across the board.

AOL SERP CTR Study – Page Depth

• 1st page: 4,425,226 89.82%
• 2nd page: 501,397 10.18%

Based on that, we assign a page-weight coefficient or multiplier (if !1, 10^-x)

Observing the Obvious:  1 in 100 searches go to Page 3.  1 in 100 million searches go to Page 10?

Sample Scoring

Using different algorithms, here are some sample SERPrints and their corresponding scores.  Which score do you think properly reflects that SERPs rankings?

Algo A = (SEw + SERPw) x Pw

Algo B = SEw x SERPw x Pw

This raw score is then multiplied by a coefficient to produce a 100-point system.   We get the coefficient by first calculating a {1,1,1} SERPrint then by dividing out the raw score by 100 to get the coefficient.  For A, it was 0.45425958 and for B, it was .025331969.

Sample Values

Example 1

{2,3,15}

Algo A = SERP SCORE 47.45
Algo B =  SERP SCORE 24.78
Example 2

{1,0,50}

Algo A = 48.86
Algo B =  69.69

Example 3

{70,2,1}

Algo A = 37.41
Algo B =  17.18

Assumptions

Here are the assumptions that I used for this:

• The 6.3% left over from other search engines in SEw is irrelevant so discard it.   I teetered on this one and wanted to spread out the 6.3% proportionately across all the 3 to account for the searches but decided against it.
• People click the same way on any search engine, keyword, regardless of sex, gender, mood, or medium.  This could play a factor since more ‘desperate’ keywords might see deeper checks: “How to cure gonorrhea” or “How to hide a body”
• People usually display 10 results per page.
• This score does not factor in ‘keyword volume’.   While this is, by far, one of the most important factors to a true ‘SERP SCORE’, it’s outside the scope of this piece.
• Who cares about SERP scoring?  It’s just another useless statistic that means nothing. I get it.

What method would you use to calculate a SERP score?  How would you factor in KEYWORD VOLUME?

Edward Lewis Has Earned the Right to be Called : SEO Hall Monitor

*Final Update*

Shutting down a site as soon as the community questions is a sign of a scam. The same way search engine submission is a scam, in our industry at the moment, SEO Verification is a scam. There is no real body or authority that is geared to give you that stamp. For a few hours the site was turned to an “April Fool’s” prank, but I dont buy it, and neither did Alan Bleiweiss. In fact the whole proposition was seen as so insane, it managed to bait The Mad Hat to post for the first time in years. Todd Mintz took out a few minutes to point out:

Seriously folks, if you want to verify the quality of a SEO vendor, ask for references. Read their blog posts. Look at their LinkedIn page and see how much a part of the SEO community they are.

SEOAly just labelled the service a Verified Failure, and hit the nail on the head. On the other hand, Kim took a very balanced and mature look at what happened, and where Charles Preston made a mistake – in underestimating the industry which he aimed to “verify”. Geeks on Steroids have now grabbed the original domain that Charles Preston started the first version of this service., and is planning on making it a tribute site to verified SEO.

The site is no longer operational (archive here – thanks Sebastian!) and Charles Preston never responded to my challenge:

I will pay the $99 fee to Verified SEO if they can prove that the site and the verification process is run by “Industry Veterans”.

Binding Contract.

If there are no truths to these claims, wait for a whole host of SEOBullshit Articles.

Clearing The Air on VerifiedSEO.com

Peek a Boo I see you!

Peace out.

****************Previous UPDATES BELOW***************

Dr Pete Gets Unverified for SEO

You Have to Love Dr Pete.

*Update 1* Get your Own “Say No To Verified SEO” Badge here.

*Update 2* Nick Wilsdon Hits a hard and simple argument:

Nick: It says an awful lot about your claim to be ‘industry veterans’ that you

(a) don’t know who Edward is without Googling his name and

(b) you just called him Ed.We are, however, most certainly qualified to verify an SEM vendor’s past success record.

VerifiedSEO: Actually anybody could do that, it just helps that we have worked in the SEM industry for a long time.

Nick: What absolute BS.  So you’re saying you don’t need to be qualified to investigate the work of another professional. So we can grab someone off the street to verify the work of an accountant, lawyer or doctor? The body that certifies professionals has to have some kind of credibility within that profession.

*Update 3* Jill Whalen uncovers some interesting “rankings” that one of the founders boasted about. To quote Jill:

Can’t imagine how Verifed SEO could judge SEO when rankings they got for their clients were for kws nobody searches for #guineapigseo

• consumer debt collection software
• debt collection management system
• online debt collection software

According to Google’s keyword research tool, even now, a few years later when there’s more competition, there are exactly NO searches at Google for those keywords. (via Sphinn)

We are in your Reputation.

*Update* 4: And the Verified SEO Story Hit TweetMeme Home Page

This post is served best as a “double-scoop” on top of Goosh’s Hit & Run Snake Oil Salesmen. And, the opening parlays are much the same:

The Email

Subject: Great Organic Traffic Results, NO COST

(actual email: names have been scrubbed)

I have SEO & link building services company, have team professional and I’ve Big site list where I can put your link on them, all my site different IPs and hosted based from USA. I’m accept services On Page/Off Page with following SERPs optimization, Keyword Optimization, Blog Posting, One Way Links, Blog Commenting, Article Writing, Link Wheel, Forum Posting, Article Submission, Logo/Banner Creation and more

I accept SEO services per keyword, This method is very important to improve SERPs in google, I will make your keyword in the first page in SERPs google, and will not stop until your keyword in the first page I will guarantee within 1-2 months your keywords will be in the first page in SERPs

Price : $100 Weekly /keyword

No Such Thing as a Free Lunch

There is this notion that’s been going around that SEO services ought to be cheap, since, apparently, we SEOs are so damn superfluous and autodidactic (self-taught; for the most part). News flash: all SEOs are NOT CREATED EQUAL.

If you opt in for something like the above, then please know, after your site’s been burned to the ground and sandboxed to the nether regions of the web, you got what you deserved. Period. And, if you were working with a reputable SEO, and dropped them in favor of economic savings, that goes double for you.

If and when a good SEO gets your site, please understand that is nigh impossible to un-fuck the damage a hack job can do to your site. It’ll end up cost you more in the long run to play it on the cheap now.

Look, if you want good help, it costs. It won’t be the most expensive service you get online, but the business community has to grasp the notion that this is no longer just a “fad” and optional. So do yourself a favor and create a filter to junk these emails.

If you want good, reliable, and solid SEO and SEM help, ask people whom you trust who they work with. Nothing will inspire more confidence and trust than a WOM recommendation. And, if you’re the first of your ilk to start SEO services, start reading search marketing blogs and news sites. Not only will this help guard you against the snake oil folks of the world, but you’ll end up with a list of names and firms that have built a trustworthy reputation.

You know how it goes:

Salesman: Hi Sir, I’m calling today about your website

Owner:  Oh, yes?

Salesman:  How would you like to put your website in front of thousands of potential visitors every day?

Owner:  Why, that sounds amazing, but how can you do that?

Salesman:  With our secret techniques we can get you #1 position on Google this afternoon!

Owner:  Wow, you must be good!

And we all know the outcome of this fabrication:  A Pay Per Click campaign that will exhaust itself faster than you can say “Whoopahh”!

It’s companies like this that annoy me so much because they are taking valuable budget away from a client where they could be easily investing in their long term rankings which will ultimately last a hell of a lot longer than the second it takes to max out your credit card!

Who’s The Guilty Parties?

The company offering this as a long-term, sustainable solution; simple as that!  The regular perpetrators are print & media agencies, small web design companies & IT support companies who have a web designer in their midst.

Don’t get me wrong, PPC has its place in high yielding, quick-reaction and absolute targeting, but when it comes to snakeoil sales companies selling this shit as SEO it really does boil my piss.  PPC can be used VERY effectively providing there is a profitable return-on-investment, but to be sold purely as a guaranteed #1 solution is just insane – especially in a very competitive industry where CPC’s are astronomical for some people.

I thought this tactic had died out in the early 2000’s but apparently enough time has passed that there are enough gullible victims at the end of a phone line once again :(

I’m not #1 anymore, what do I do?

I would honestly hate to be the account managers on the end of the phone having to explain it to these poor souls that they have to spend, spend & spend money to remain on the 1st page and then be questioned why their competitor is always there on the 1st page.  I imagine it goes like this:

Account Manager: Oh, that’s a Natural Listing

Owner:  A what?

Account Manager: Through SEO

Owner:  That’s what I just bought

Account Manager: No, you bought Sponsored Listing where you pay to be there and once your budget runs out you are removed

Owner:  What?

All I can hope is that the clean cut, honorable SEO’s get to these companies first so we can at least educate them on the difference before they spend their yearly budget on a months 1st page listing with these snakeoil salesmen – and if they do happen to get stung, I hope they can make their money from the deal and prove me wrong!

But I doubt that will happen.

To make a long and redundant story short (I shit you not, more here) this whole thing began when Google purchased a company called Applied Semantics. This company was stated at the time to be of interest to Google for their new ad matching technology dubbed ‘AdSense’.

Yup, that’s right. Does that sound familiar to you Mr. LSI SEO dumbnuts? Ad-Sense… say it faster… AdSense. But oh no… some nearly 6 years later and there are still those that missed it. There is still crap hat SEO services and chatter on forums of the mystical LSI and Google.

Just to get it home… from the press release;

They spoke of their interest in, “Applied Semantics’ AdSense product that enables web publishers to understand the key themes on web pages to deliver highly relevant and targeted advertisements.”

Get that? It was for AdSense that LSI was originally incorporated jack ass.

I can hear the nay-sayers now

“But it could have been used for the organic search”

No no no no. STFU and sit back down skippy. How do you explain Anna Patterson and Phrase Based IR purchase less than a year later? Those were some pretty interesting semantic analysis methods (often confused with LSI)… Weren’t the Google engineers marvelling at PLSA, HTMM and even LDA over on the Google research blog a year or so back?

Confused? More here at your leisure;

1. Phrase based indexing and retrieval methods
2. Probabilistic latent semantic analysis
3. Latent Dirichlet allocation
4. Hidden Topic Markov Models

Then you might want to read this Google Research post from 2007…just for shits and giggles.

So? What about those? Can someone selling this fucking Google LSI bullshit please drop by and share the data with me that points to the assertion yer making? On second thought, forget it. Just stop using the term Latent Semantic Indexing/Analysis and we’ll all be better for it

Say it along with me; Semantic Analysis

What SHOULD SEOs refer to it as? No idea man… if I was that smart I’d work at Google not wallowing here in misery. Just start saying that, “Google uses semantic analysis”. What flavour? We don’t know… but at least we’re not bullshitting people right?

At the end of the day people seemed to have gotten lost somewhere between 2003 and 2006 when the ‘Google LSI’ crap really picked up speed. It went from being the incarnation of AdSense to being the semantic tool of choice in the regular index/SERPs. Even the history doesn’t agree with that (shit, even the phrase based stuff and hiring on Anna Patterson was shortly after).

And there lies much of the problem, SEOs not only having a low level of information retrieval knowledge, but not understanding search history either.

The next time some dumbass SEO starts talking about LSI, please, I implore you, send them to this post. Or hey, just shake them about some and tell them to stop making the rest of us look like a bunch of ignorant snake oil shit heads.

Thanks…

Crap worth nuking!!

How Does Google Use Latent Semantic Indexing (LSI)? – Sounds pretty says nothing…

What Is Google LSI - References Anna Pattersons PaIR patents as LSI… duh

Google Debunks LSI – Then Shows You How To Do It - Understands it doesn’t scale well, but still cashes in on the term

Warning – “Advanced” SEO Technique DOES NOT WORK – Talks about LSI but uses the Anna Patterson patents as the example; FAIL

Is LSI Dead? - Clueless… doesn’t get it.

The Google LSI Handbook – what can I say… WOW… assholes.

…there’s lot’s more…but I can’t stomach it…. ugh…

People only hear three things when it comes to SEO and these are:

• Long Term Effort
• Quality
• Cost

Frankly that’s exactly what they DO NOT want to hear as it all adds up to “expensive“.  No bullshit.  Most people want faster results at super cheap prices and instead prefer to spend serious amounts of time and waste silly amounts of money looking for ways to take advantage of the bullshit SEO vendors promises of wealth, fame and fortune if they just by a book and become an instant SEO master blasta and it goes a little something like this.

“Sign up for only 25-99, reduced from 59-99!

Spread your name through our magic triangle of interlinked sites and gain exposure to hundreds of thousands of people with our amazing and highly reputable SEO system.

Analyse your site and fix all those SEO problems in seconds with one click and see your site at number one for some obscure keywords that no one ever types, but hell! At least it looks good!

Tell your friends and make them jealous so they buy this bullshit as well!”

This kind of crap is then followed up by a mangled list of crap testimonials and “certifications” from even more obscure bodies that are probably owned by the very same group or company and which they use to validate this super cool system.  Talk about third party and independent opinion?  Get real!  Why bother, no one in their right mind would put their name against this drivel.  no one real that is!  So what, lets just make it up!  No one checks this anyway and the promise of fast wealth is far to tempting.

Like doing a quick Google search for domain name ownership on WHOIS takes to much effort or taping in the name of the vendor and adding “are they legit” to the end wont bring back some type of opinion from another person or body who wont be afraid to tell they real story about this bullshit.  Sheesh!

Heres some more bullshit to chew on with these SEO books, lets dissect this bit by bit:

“Finally, A Powerful Set Of Search Engine Optimization Tools So Easy To Use That You
Will Blow By Your Competitors, Get Higher
Search Engine Rankings, & Make More
Profits Starting Today!…”

Right. So what your telling me here is that with this book I will kick ass and destroy all the competition, even though they may have the very same book and be doing what it says as well, word for word. Lets not forget the fact that the minute its released its out of date and also lets just assume for one second that every single website on the planet is exactly the same so this inane and bollocks strategy will work as well for companies that sell footballs as one that sells multi million dollar villas.

Aye Right.

“Master Resell Rights Included”

Oh right, that makes it ok then.  Thanks for that ya douche.

“The Search Engine Game
Has Been Exoposed And Is
NOW Wide Open!
Are you tired of spending loads of money trying to get search engine traffic?”

Right, here is the killer. This is the line that gets my blood boiling every single time and it just stinks of bullshit. In fact, for every intelligent person that reads this I’m not even gonna bother explaining just how much crap is in these four lines, it should be immediately obvious.

Maybe you’re just tired of spending hundreds of hours trying to tweak and change your site just right so that you can hope to get a decent ranking.

Your crap at this. Get over it and go do something your good at.

Truthfully, have any of this paid off in the way that you pictured they would?

Chances are, they haven’t but don’t fret because there is good news!

To find out how to start getting the rankings you want and deserve, read on…”

Well… if you have spent hundreds of hours tweaking your site and you are not getting a decent ranking then maybe you should stop feckin about and go and write some decent content that people may actually want to read, or alternatively, just give up, cause at this stage you must be completely useless. In fact, think about hiring someone who knows what they are doing and stop spending your money on this rubbish and wasting your life.

Bloody Hell.