data.gov icon indicating copy to clipboard operation
data.gov copied to clipboard

SEO loss on catalog

Open jbrown-xentity opened this issue 4 years ago • 11 comments

Once the new catalog was released in Feb 2021, the number of page views from google decreased 70-80%. We need to investigate why this happened (ckanext-dcat misconfiguration? sitemap error? robots.txt change?) and make the necessary fixes to be indexed by Google better.

User Story

In order to understand significant changes in Data.gov page views in Google Analytics, Data.gov PMO needs to investigate possible causes and make necessary fixes.

Acceptance Criteria

  • [ ] GIVEN the significant drops in page views that started to occur in early 2021
    THEN Data.gov PMO has an identified cause
    AND has identified necessary actions to correct any errors or misconfiguration.

Background

There has been significant drops in page views, when comparing the same month from year to year, in GA. See details at https://docs.google.com/spreadsheets/d/14zlpisixgr4_KrEwmRlWQsVk-0tE_o-5eRfGRzLNHqA/edit?usp=sharing

Sketch

  • [ ] Review numbers on Data.gov account in GA (UA-42145528)
  • [ ] Correlate with events at key times, such as launch of new version of catalog (CKAN 2.8)
  • [ ] Investigate causes including change in catalog URLs

jbrown-xentity avatar Sep 13 '21 20:09 jbrown-xentity

Could consider improving analytics, see email on 9/28/2021 at 2:23 ET. Get a new tag.

jbrown-xentity avatar Sep 28 '21 19:09 jbrown-xentity

Screen Shot 2021-10-05 at 5 00 48 PM Screen Shot 2021-10-05 at 5 01 48 PM In 2020 and prior months, google was top source, and sometime in February-March 2021, google drops off significantly and top source is direct

hkdctol avatar Oct 05 '21 21:10 hkdctol

I looked through this time period, and there was not an obvious drop off point in the switching places between google and direct. For instance it did not appear tied to the launch of the new catalog (2/5/21)

hkdctol avatar Oct 12 '21 18:10 hkdctol

There's also an email today about GA 4: https://analytics.google.com/analytics/web/?utm_campaign=2021-q4-gbl-all-gafree&utm_source=google-growth&utm_medium=email&utm_content=ga4-sot-ga4-setup-assistant-nonadv#/a42542568w72309185p0/admin/ga4-setup-assistant

hkdctol avatar Oct 12 '21 18:10 hkdctol

The switch seems to have happened in mid-February 2021

Screen Shot 2021-10-19 at 4.05.50 PM.pngScreen Shot 2021-10-19 at 4.05.30 PM.png

hkdctol avatar Oct 19 '21 20:10 hkdctol

Situation remains the same in recent analytics view: Screen Shot 2022-01-04 at 6.33.12 PM.png

hkdctol avatar Jan 04 '22 23:01 hkdctol

Searching for some popular datasets, such as GSA federal real property, or GSA per diem, the GSA pages appear at top of search results, catalog.data.gov version not found until second page for the real property dataset, and not found at all for per diem.

hkdctol avatar Jan 04 '22 23:01 hkdctol

Everyone on the team should have access to Google Search Console now for data.gov. There's a good DAP training on GSC at https://www.youtube.com/watch?v=uuP0FAHOrz8&list=PLd9b-GuOJ3nEz1NYl66orgVZIu17laKba&index=48

hkdctol avatar Jan 07 '22 23:01 hkdctol

The training points out parts of Google Search Console to check for things that might prevent Google indexingScreen Shot 2022-01-10 at 11.26.19 AM.png

hkdctol avatar Jan 10 '22 17:01 hkdctol

The training also emphasized submitting a sitemap in Google Search Console.

hkdctol avatar Jan 10 '22 17:01 hkdctol

A month by month comparison is in this doc

hkdctol avatar Aug 09 '22 14:08 hkdctol

With recent sitemap publishing, we hope that this trends upward soon. If not, we should consider finding expert help in guiding how to improve SEO, not investigate how/what we broke. In the case of long term improvements, this should be considered an epic where details would need to be filled in during some type of planning session.

jbrown-xentity avatar Nov 29 '22 22:11 jbrown-xentity