Embed icon indicating copy to clipboard operation
Embed copied to clipboard

Make snapshot updates possible without modifying test code

Open uzulla opened this issue 5 months ago • 2 comments

Why

Updating test snapshots required modifying constants inside the test suite. This made the workflow unnecessarily awkward and error-prone:

  • Snapshot updates required editing test code.
  • Accidental commits of development-only changes were possible.
  • Numeric flags were hard to remember and not self-explanatory.
  • Cache and fixtures were controlled separately even though they are always updated together

As a result, contributors could not easily refresh snapshots, raising the barrier for maintaining or improving tests.

This PR introduces a simple environment-based switch so contributors can update snapshots without touching the codebase.


What

New environment variables

Variable Purpose
UPDATE_EMBED_SNAPSHOTS=1 Re-fetch from the network and regenerate both cache and fixtures used in tests
EMBED_STRICT_CACHE=1 Fail tests if required cache files are missing (useful for CI)

Developer workflow

Developers can now refresh snapshots with a single command:

UPDATE_EMBED_SNAPSHOTS=1 ./vendor/bin/phpunit

No code edits and no numeric flags are required.

CI updates

GitHub Actions now runs with:

EMBED_STRICT_CACHE=1

to ensure snapshot files are always present and up to date.

EMBED_STRICT_CACHE=1 ./vendor/bin/phpunit

Documentation

A new "Testing" section has been added to the README, describing how snapshots are updated and how strict mode behaves in CI.


Usage Examples

# Update snapshots for a specific test
UPDATE_EMBED_SNAPSHOTS=1 ./vendor/bin/phpunit --filter testYoutube

# Update all snapshots
UPDATE_EMBED_SNAPSHOTS=1 ./vendor/bin/phpunit

Execute result

Update success on Linux, please open details.

env(docker)

# Dockerfile.test
# docker build -f Dockerfile.test -t embed-test .
# docker run --rm embed-test
FROM php:8.4-cli

# Install curl and dependencies
RUN apt-get update && apt-get install -y \
    libcurl4-openssl-dev \
    libzip-dev \
    unzip \
    git \
    && docker-php-ext-install curl \
    && rm -rf /var/lib/apt/lists/*

# Install Composer
COPY --from=composer:latest /usr/bin/composer /usr/bin/composer

# Set working directory
WORKDIR /app

# Copy composer files
COPY composer.json composer.lock ./

# Install dependencies
RUN composer install --no-interaction --no-dev --prefer-dist

# Copy the rest of the application
COPY . .

# Set environment variable
ENV UPDATE_EMBED_SNAPSHOTS=1

# Run tests with error reporting options
CMD ["php", "-d", "error_reporting=E_ALL^E_DEPRECATED^E_STRICT", "vendor/bin/phpunit"]

Execute output

PHPUnit 9.6.29 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.4.15
Configuration: /app/phpunit.xml.dist

........................S
Save fixture: https://www.abanca.com/glS
Save fixture: https://animoto.com/play/GjsJ1gu0WDRfr4pGw12xZQS
Save fixture: http://live.amcharts.com/czNjJS
Save fixture: https://www.aol.com/video/view/pile-of-recovering-foster-kittens-is-purrfect/595fe75985eb42109b69bedb/S
Save fixture: http://jeanjean.bandcamp.com/track/coquin-l-l-phantS
Save fixture: http://codepen.io/Zhouzi/pen/JoRazPS
Save fixture: http://www.dailymotion.com/video/xy0wdS
Save fixture: http://www.deviantart.com/art/Misty-510056679S
Save fixture: http://pachunka.deviantart.com/art/Cope-145564099S
Save fixture: http://www.hookem.com/story/texas-shortstop-joe-baker-arrested-public-intoxication/S
Save fixture: http://i.imgur.com/X6rkCc5.jpgS
Save fixture: https://infogr.am/7743c36a-f3ca-4465-9a80-a8abbd5d8dc4S
Save fixture: http://output.jsbin.com/vonesu/10S
Save fixture: http://jsfiddle.net/zhm5rjnz/S
Save fixture: https://www.kickstarter.com/projects/1452363698/good-seed-craft-veggie-burgersS
Save fixture: http://www.23hq.com/Zzleeper/photo/16600737S
Save fixture: https://500px.com/photo/138251239/taganay-park-by-daniel-kordanS
Save fixture: https://pastebin.com/d4biUtRmS
Save fixture: http://media.photobucket.com/user/Ignwar/media/Album%20Deserts/MoonriseMonumentValleyUtah.jpg.html?filters[term]=sunsets&filters[primary]=imagesS
Save fixture: http://www.politico.com/story/2013/12/presidents-barack-obama-george-w-bush-second-term-101314.htmlS
Save fixture: http://polldaddy.com/poll/7012505/S
Save fixture: https://www.reddit.com/r/investing/comments/7pfpeq/buffett_on_cyrptocurrencies_i_can_say_almost_with/S
Save fixture: http://www.scribd.com/doc/110799637/Synthesis-of-Knowledge-Effects-of-Fire-and-Thinning-Treatments-on-Understory-Vegetation-in-Dry-U-S-ForestsS
Save fixture: http://www.spreaker.com/user/angelclark/angel-clark-ice-cream-tocosS
Save fixture: http://www.ted.com/talks/george_monbiot_for_more_wonder_rewild_the_world?language=en#t-689901S
Save fixture: http://he-who-photographs-rather-ok.tumblr.com/post/165326273724S
Save fixture: http://www.ustream.tv/channel/red-shoes-billiards-60803-camera-1S
Save fixture: http://www.viddler.com/v/bdce8c7S
Save fixture: http://www.wired.com/?p=2064839S
Save fixture: https://www.itmedia.co.jp/news/articles/2410/28/news159.htmlS
Save fixture: https://4pda.to/2022/12/04/406834/sostoyalsya_reliz_clown_of_duty_parodii_na_call_of_duty/S
Save fixture: https://cdn2.thecatapi.com/images/cjd.jpgS
Save fixture: https://www.soundhelix.com/examples/mp3/SoundHelix-Song-1.mp3S
Save fixture: http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4S
Save fixture: http://snipplr.com/view/72914/better-html-5-basic-starter-templateS
Save fixture: https://imageshack.com/i/ip7wO0v7jS
Save fixture: https://ideone.com/WhjntgS
Save fixture: http://play.cadenaser.com/audio/001RD010000004275766/S
Save fixture: http://slides.com/alexwalker/responsive-svg/S
Save fixture: https://archive.org/details/dn2015-0220_vidS 65 / 81 ( 80%)
S
Save fixture: https://www.meetup.com/es/GPUL-Labs/events/248885422/S
Save fixture: https://www.pinterest.com/pin/106890191127977979/S
Save fixture: https://en.wikipedia.org/wiki/Albert_EinsteinS
Save fixture: https://vimeo.com/235352744S
Save fixture: http://wordpress.tv/2013/09/06/dave-ross-optimize-image-files-like-a-pro/S
Save fixture: https://www.flickr.com/photos/desescribir/sets/72157650686499888SS
Save fixture: https://gist.github.com/oscarotero/7749998S
Save fixture: https://www.google.es/maps/place/Tordoia,+A+Coru%C3%B1a/@43.0871207,-8.5710004,12z/data=!3m1!4b1!4m2!3m1!1s0xd2ef4006f1ef489:0x404f58273ca55a0S
Save fixture: https://soundcloud.com/zedsdead/zeds-dead-twin-shadow-lost-you-feat-dangelo-lacyS
Save fixture: https://open.spotify.com/album/7s66wU1XJ2NsUuWM2NKiUVS
Save fixture: https://www.twitch.tv/videos/72749628S
Save fixture: https://twitter.com/pepephone/status/436461658601713664S
Save fixture: https://www.tiktok.com/@a3noticias/video/6806030056956251397S
Save fixture: http://www.youtube.com/watch?v=eiHXASgRTcAS                                                  81 / 81 (100%)
Save fixture: https://www.bbc.co.uk/news/uk-54222286

Time: 01:54.261, Memory: 30.14 MB

There were 57 skipped tests:

1) Embed\Tests\PagesTest::testPages with data set #0 ('https://www.abanca.com/gl')
Skipped assertion for https://www.abanca.com/gl

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

2) Embed\Tests\PagesTest::testPages with data set #1 ('https://animoto.com/play/GjsJ...w12xZQ')
Skipped assertion for https://animoto.com/play/GjsJ1gu0WDRfr4pGw12xZQ

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

3) Embed\Tests\PagesTest::testPages with data set #2 ('http://live.amcharts.com/czNjJ')
Skipped assertion for http://live.amcharts.com/czNjJ

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

4) Embed\Tests\PagesTest::testPages with data set #3 ('https://www.aol.com/video/vie...9bedb/')
Skipped assertion for https://www.aol.com/video/view/pile-of-recovering-foster-kittens-is-purrfect/595fe75985eb42109b69bedb/

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

5) Embed\Tests\PagesTest::testPages with data set #4 ('http://jeanjean.bandcamp.com/...-phant')
Skipped assertion for http://jeanjean.bandcamp.com/track/coquin-l-l-phant

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

6) Embed\Tests\PagesTest::testPages with data set #5 ('http://codepen.io/Zhouzi/pen/JoRazP')
Skipped assertion for http://codepen.io/Zhouzi/pen/JoRazP

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

7) Embed\Tests\PagesTest::testPages with data set #6 ('http://www.dailymotion.com/video/xy0wd')
Skipped assertion for http://www.dailymotion.com/video/xy0wd

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

8) Embed\Tests\PagesTest::testPages with data set #7 ('http://www.deviantart.com/art...056679')
Skipped assertion for http://www.deviantart.com/art/Misty-510056679

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

9) Embed\Tests\PagesTest::testPages with data set #8 ('http://pachunka.deviantart.co...564099')
Skipped assertion for http://pachunka.deviantart.com/art/Cope-145564099

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

10) Embed\Tests\PagesTest::testPages with data set #9 ('http://www.hookem.com/story/t...ation/')
Skipped assertion for http://www.hookem.com/story/texas-shortstop-joe-baker-arrested-public-intoxication/

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

11) Embed\Tests\PagesTest::testPages with data set #10 ('http://i.imgur.com/X6rkCc5.jpg')
Skipped assertion for http://i.imgur.com/X6rkCc5.jpg

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

12) Embed\Tests\PagesTest::testPages with data set #11 ('https://infogr.am/7743c36a-f3...5d8dc4')
Skipped assertion for https://infogr.am/7743c36a-f3ca-4465-9a80-a8abbd5d8dc4

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

13) Embed\Tests\PagesTest::testPages with data set #12 ('http://output.jsbin.com/vonesu/10')
Skipped assertion for http://output.jsbin.com/vonesu/10

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

14) Embed\Tests\PagesTest::testPages with data set #13 ('http://jsfiddle.net/zhm5rjnz/')
Skipped assertion for http://jsfiddle.net/zhm5rjnz/

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

15) Embed\Tests\PagesTest::testPages with data set #14 ('https://www.kickstarter.com/p...urgers')
Skipped assertion for https://www.kickstarter.com/projects/1452363698/good-seed-craft-veggie-burgers

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

16) Embed\Tests\PagesTest::testPages with data set #15 ('http://www.23hq.com/Zzleeper/...600737')
Skipped assertion for http://www.23hq.com/Zzleeper/photo/16600737

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

17) Embed\Tests\PagesTest::testPages with data set #16 ('https://500px.com/photo/13825...kordan')
Skipped assertion for https://500px.com/photo/138251239/taganay-park-by-daniel-kordan

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

18) Embed\Tests\PagesTest::testPages with data set #17 ('https://pastebin.com/d4biUtRm')
Skipped assertion for https://pastebin.com/d4biUtRm

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

19) Embed\Tests\PagesTest::testPages with data set #18 ('http://media.photobucket.com/...images')
Skipped assertion for http://media.photobucket.com/user/Ignwar/media/Album%20Deserts/MoonriseMonumentValleyUtah.jpg.html?filters[term]=sunsets&filters[primary]=images

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

20) Embed\Tests\PagesTest::testPages with data set #19 ('http://www.politico.com/story...4.html')
Skipped assertion for http://www.politico.com/story/2013/12/presidents-barack-obama-george-w-bush-second-term-101314.html

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

21) Embed\Tests\PagesTest::testPages with data set #20 ('http://polldaddy.com/poll/7012505/')
Skipped assertion for http://polldaddy.com/poll/7012505/

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

22) Embed\Tests\PagesTest::testPages with data set #21 ('https://www.reddit.com/r/inve..._with/')
Skipped assertion for https://www.reddit.com/r/investing/comments/7pfpeq/buffett_on_cyrptocurrencies_i_can_say_almost_with/

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

23) Embed\Tests\PagesTest::testPages with data set #22 ('http://www.scribd.com/doc/110...orests')
Skipped assertion for http://www.scribd.com/doc/110799637/Synthesis-of-Knowledge-Effects-of-Fire-and-Thinning-Treatments-on-Understory-Vegetation-in-Dry-U-S-Forests

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

24) Embed\Tests\PagesTest::testPages with data set #23 ('http://www.spreaker.com/user/...-tocos')
Skipped assertion for http://www.spreaker.com/user/angelclark/angel-clark-ice-cream-tocos

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

25) Embed\Tests\PagesTest::testPages with data set #24 ('http://www.ted.com/talks/geor...689901')
Skipped assertion for http://www.ted.com/talks/george_monbiot_for_more_wonder_rewild_the_world?language=en#t-689901

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

26) Embed\Tests\PagesTest::testPages with data set #25 ('http://he-who-photographs-rat...273724')
Skipped assertion for http://he-who-photographs-rather-ok.tumblr.com/post/165326273724

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

27) Embed\Tests\PagesTest::testPages with data set #26 ('http://www.ustream.tv/channel...mera-1')
Skipped assertion for http://www.ustream.tv/channel/red-shoes-billiards-60803-camera-1

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

28) Embed\Tests\PagesTest::testPages with data set #27 ('http://www.viddler.com/v/bdce8c7')
Skipped assertion for http://www.viddler.com/v/bdce8c7

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

29) Embed\Tests\PagesTest::testPages with data set #28 ('http://www.wired.com/?p=2064839')
Skipped assertion for http://www.wired.com/?p=2064839

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

30) Embed\Tests\PagesTest::testPages with data set #29 ('https://www.itmedia.co.jp/new...9.html')
Skipped assertion for https://www.itmedia.co.jp/news/articles/2410/28/news159.html

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

31) Embed\Tests\PagesTest::testPages with data set #30 ('https://4pda.to/2022/12/04/40..._duty/')
Skipped assertion for https://4pda.to/2022/12/04/406834/sostoyalsya_reliz_clown_of_duty_parodii_na_call_of_duty/

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:64

32) Embed\Tests\PagesTest::testImageFile
Skipped assertion for https://cdn2.thecatapi.com/images/cjd.jpg

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:69

33) Embed\Tests\PagesTest::testAudioFile
Skipped assertion for https://www.soundhelix.com/examples/mp3/SoundHelix-Song-1.mp3

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:74

34) Embed\Tests\PagesTest::testVideoFile
Skipped assertion for http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:79

35) Embed\Tests\PagesTest::testSnipplr
Skipped assertion for http://snipplr.com/view/72914/better-html-5-basic-starter-template

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:84

36) Embed\Tests\PagesTest::testImageShack
Skipped assertion for https://imageshack.com/i/ip7wO0v7j

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:89

37) Embed\Tests\PagesTest::testIdeone
Skipped assertion for https://ideone.com/Whjntg

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:94

38) Embed\Tests\PagesTest::testCadenaSer
Skipped assertion for http://play.cadenaser.com/audio/001RD010000004275766/

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:99

39) Embed\Tests\PagesTest::testSlides
Skipped assertion for http://slides.com/alexwalker/responsive-svg/

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:104

40) Embed\Tests\PagesTest::testArchiveOrg
Skipped assertion for https://archive.org/details/dn2015-0220_vid

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:109

41) Embed\Tests\PagesTest::testInstagram
Environment variable `INSTAGRAM_TOKEN` must be provided to test instagram. See https://developers.facebook.com/docs/instagram/oembed/

/app/tests/PagesTest.php:118

42) Embed\Tests\PagesTest::testMeetup
Skipped assertion for https://www.meetup.com/es/GPUL-Labs/events/248885422/

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:124

43) Embed\Tests\PagesTest::testPinterest
Skipped assertion for https://www.pinterest.com/pin/106890191127977979/

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:129

44) Embed\Tests\PagesTest::testWikipedia
Skipped assertion for https://en.wikipedia.org/wiki/Albert_Einstein

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:134

45) Embed\Tests\PagesTest::testVimeo
Skipped assertion for https://vimeo.com/235352744

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:139

46) Embed\Tests\PagesTest::testWordPress
Skipped assertion for http://wordpress.tv/2013/09/06/dave-ross-optimize-image-files-like-a-pro/

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:144

47) Embed\Tests\PagesTest::testFlickr
Skipped assertion for https://www.flickr.com/photos/desescribir/sets/72157650686499888

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:149

48) Embed\Tests\PagesTest::testFacebook
Environment variable `FACEBOOK_TOKEN` must be provided to test facebook. See https://developers.facebook.com/docs/plugins/oembed

/app/tests/PagesTest.php:159

49) Embed\Tests\PagesTest::testGithub
Skipped assertion for https://gist.github.com/oscarotero/7749998

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:165

50) Embed\Tests\PagesTest::testGoogle
Skipped assertion for https://www.google.es/maps/place/Tordoia,+A+Coru%C3%B1a/@43.0871207,-8.5710004,12z/data=!3m1!4b1!4m2!3m1!1s0xd2ef4006f1ef489:0x404f58273ca55a0

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:173

51) Embed\Tests\PagesTest::testSoundCloud
Skipped assertion for https://soundcloud.com/zedsdead/zeds-dead-twin-shadow-lost-you-feat-dangelo-lacy

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:180

52) Embed\Tests\PagesTest::testSpotify
Skipped assertion for https://open.spotify.com/album/7s66wU1XJ2NsUuWM2NKiUV

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:187

53) Embed\Tests\PagesTest::testTwitch
Skipped assertion for https://www.twitch.tv/videos/72749628

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:193

54) Embed\Tests\PagesTest::testTwitter
Skipped assertion for https://twitter.com/pepephone/status/436461658601713664

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:200

55) Embed\Tests\PagesTest::testTikTok
Skipped assertion for https://www.tiktok.com/@a3noticias/video/6806030056956251397

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:206

56) Embed\Tests\PagesTest::testYoutube
Skipped assertion for http://www.youtube.com/watch?v=eiHXASgRTcA

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:211

57) Embed\Tests\PagesTest::testBBCNews
Skipped assertion for https://www.bbc.co.uk/news/uk-54222286

/app/tests/PagesTestCase.php:134
/app/tests/PagesTest.php:219

OK, but incomplete, skipped, or risky tests!
Tests: 81, Assertions: 47, Skipped: 57.

on macOS error sample

But failed on MY mac. It’s probably due to macOS resource limitations, and I could not solve fails. But It succeeds on Linux, so I don’t think the code itself is incorrect.

Save fixture: http://jsfiddle.net/zhm5rjnz/
 ↩ Pages with data set #14 [1139.75 ms]
   │
   │ Skipped assertion for https://www.kickstarter.com/projects/1452363698/good-seed-craft-veggie-burgers
   │
   │ /Users/zishida/dev/PHpEmbed-Embed/tests/PagesTestCase.php:134
   │ /Users/zishida/dev/PHpEmbed-Embed/tests/PagesTest.php:64
   │

Save fixture: https://www.kickstarter.com/projects/1452363698/good-seed-craft-veggie-burgers
 ✘ Pages with data set #15 [7993.72 ms]
   │
   │ Embed\Http\NetworkException: getaddrinfo() thread failed to start
   │
   │ /Users/zishida/dev/PHpEmbed-Embed/src/Http/CurlDispatcher.php:200
   │ /Users/zishida/dev/PHpEmbed-Embed/src/Http/CurlDispatcher.php:160
   │ /Users/zishida/dev/PHpEmbed-Embed/src/Http/CurlDispatcher.php:49
   │ /Users/zishida/dev/PHpEmbed-Embed/src/Http/CurlClient.php:35

uzulla avatar Nov 20 '25 04:11 uzulla

Thank you for waiting. It’s now ready for review.

uzulla avatar Nov 24 '25 05:11 uzulla

Thank you for waiting. It’s now ready for review.

Very well, then I'll proceed to review

Vitorinox avatar Nov 25 '25 00:11 Vitorinox