TECH SEO CONNECT 2024, Raleigh, NC - My Notes

Summary Highlights:

SEO Software Gaps and Limitations

1. Current SEO Tools Miss the Mark: Many popular tools, particularly content optimization tools, still rely on outdated models. Tools that score content purely based on keywords (TF-IDF) and comparisons to the top 20 pages fall short because they fail to account for Google's shift to semantic search and contextual relevance.

- Google uses phrase-based indexing and other models that look at co-occurring keywords and how they relate across a broad set of queries. Tools that don’t account for this fail to optimize for what Google actually considers important.

2. How Google Truly Understands Content:

- Vector embeddings: Search engines like Google no longer just count keywords but use embeddings to map content in a multidimensional space where relevance is quantified based on proximity. This shift to semantic relevance allows Google to understand nuances and context, which is not fully captured by many SEO tools.

- Hybrid retrieval models: Google combines lexical matching (keywords) with semantic understanding (context), allowing pages to rank for terms without explicitly using them. This explains why some results might rank despite not having exact keyword matches in titles or H1s.

3. Dense Retrieval and AI Overviews:

- Google's ability to rank down to the sentence or paragraph level, scoring aspects of content separately, creates challenges for SEOs who rely on outdated metrics. Google uses multi-aspect dense retrieval, meaning that even fragments of a page could be relevant for ranking.

The Evolution of Search Algorithms

1. From TF-IDF to Word Embeddings:

- Traditional models like TF-IDF are based on counting word frequency, but with advancements like Word2Vec and later BERT, search engines can understand the meaning of words in context. This shift powers modern SEO strategies like topic modeling and relevance analysis.

- BERT vs. Word2Vec: While Word2Vec allowed for understanding relationships between words, BERT brought contextual understanding, allowing Google to differentiate between similar words used in different contexts (e.g., "bank" in "riverbank" vs. "financial bank").

2. Transformer Models and Generative AI:

- The rise of Transformer models (the "T" in BERT and ChatGPT) brought even more advanced language understanding. These models allow Google to score and rank content based on deep semantic relationships rather than just surface-level keyword matching.

User Behavior and Ranking

1. Click Metrics and Behavior:

- Click data has long been a debated factor in SEO. Despite Google's public denials, leaked information (and DOJ testimony) confirms that user behavior, particularly click metrics like last longest click (users staying on a site without returning), significantly influences rankings.

- Reranking and Learning to Rank (LTR): Google adjusts rankings based on user behavior. Reranking algorithms monitor user engagement and satisfaction signals (clicks, hovers) to refine results.

2. The Role of Clickstream Data:

- King emphasizes that Google’s advantage is its vast clickstream data. Tools like SEMrush Trends and SimilarWeb provide a glimpse into this data, but no SEO tools fully harness the insights that Google has.

 

Embeddings and SEO Strategy

1. Embeddings for Sites and Authors:

- Google creates site-level embeddings, where the entire content of a site is represented as a single vector. This allows Google to understand what a site is about holistically, and determine how well individual pages align with that core topic.

- Author embeddings: Similarly, Google averages an author’s body of work, allowing the search engine to determine the relevance and trustworthiness of a piece of content based on the author’s established expertise.

​

2. Information Gain:

- Information Gain is a key factor in ranking. This metric assesses how much new, unique information a page provides compared to other pages on the same topic. SEO tools rarely measure this, though it’s a crucial aspect of search rankings.

- King provides a formula to calculate Information Gain using mutual information and entity extraction, allowing SEOs to measure how much novel content their pages offer.

​

Closing Gaps with Custom Tools

1. Developing Better SEO Metrics: King has developed several custom Python tools and Google Colabs to address the gaps in existing SEO software. These tools help compute:

- Site Focus Scores: Measure how well each page aligns with the overall focus of the website.

- Content Audits: Assess the potential of content for optimization using a composite metric called Content Potential Rating.

- Embeddings-based Audits: These tools analyze entire websites and individual pages to determine how semantically aligned they are with their core topics.

​

2. The Search Telemetry Project:

- King is working on the Search Telemetry Project, aiming to provide more comprehensive SEO metrics that align with what Google actually measures. This initiative will offer open-source data and community-driven tools to better track user behavior, content decay, and click metrics.

​

The Future of SEO Tools

1. Standardization:

- King advocates for standardizing SEO tools and data outputs. He points out the discrepancies between tools that provide different formats, making it difficult to switch platforms or integrate data. Standardizing would allow for more fluid use of SEO software across the industry.

​

2. Open-Source SEO Data:

- King proposes creating an open-source index of rankings, embeddings, and link data that the SEO community can access for free. Inspired by the Majestic model, he envisions a community-powered initiative that democratizes access to critical SEO data.

Video: https://www.youtube.com/watch?v=4NpdxySuEEg