Thumbnail image research thread

Up for a real challenge? Fire up your http header sniffers, maybe even wireshark, and let's find out where the thumbnails are kept for video hosts.

YouTube is done, so let's find the rest.

(NOTE: if you are trying to paste code, replace "<" with "&lt;" and ">" with "&gt;"
Also, replace any ampersands with &amp; but don't do this if the code has already html friendly. -- KTHX)


(these are done)
YouTube

Google Video / MySpace video
LiveLeak.com / theOnion.com
MetaCafe / Break.com
iFilm video / DailyMotion
Revver / Vimeo
Yahoo! video / ComedyCentral
VideoJug / vSocial / ForaTV


[admin edit]
Crossed out video hosts that are now supported.

joedirt says...

Good luck with google. They seem to use a checksum key. But, you can scrape the second <img tag on the individual video page.

It looks like:
<img src="http://video.google.com/ThumbnailServer2?app=vss&contentid=b7fa21520a2d48c0 &offsetms=0&itag=w160&lang=en&sigh=s3Qs4w2d0RlFDHLM1czSVVm8_mk" alt="" style="border: none; margin-left: 0px; margin-top: 0px; width: 100px; height: 75px;" class="searchresultimg">

You could probably scrape the line with "searchresultimg"

----
They also have different images for 0%, 20%, 40%, etc.
See the start time, but again, super generated key thingy.
http://video.google.com/ThumbnailServer2?app=vss&contentid=fc62d88d323c031d &offsetms=0&itag=w160&lang=en&sigh=neyJwepe638355CYUf2P53L1hUE
http://video.google.com/ThumbnailServer2?app=vss&contentid=fc62d88d323c031d &offsetms=195000&itag=w160&lang=en&sigh=8bwJtIoc24Mzt5nPH-I9EPeEYEc

joedirt says...

Just asking, would it be too hard to design a .flv downloader script, that saves the .flv temporarily and grabs a screen shot using mencoder type stuff.

maybe there is some debian stuff to do flash to stdout...

joedirt says...

LiveLeak also generates thumbs as they are uploaded in numerical order with a checksum, and there is no way to scrape the thumbnail from the video page. The only way I can think of scraping the img code is if you viewed "videos by username" and just started grabbing the thumbnails from there until you hit the right one.

http://www.liveleak.com/view?i=371_1188426824
http://cdn-01.liveleak.com/liveleak/thumbs/2007/Aug/29/0702570c113482_1_thumb.jpg

http://www.liveleak.com/view?i=77b_1188430967
http://cdn-01.liveleak.com/liveleak/thumbs/2007/Aug/29/2642b42c113517_1_thumb.jpg

http://www.liveleak.com/view?i=e0f_1188431929
http://cdn-01.liveleak.com/liveleak/thumbs/2007/Aug/29/3a0b8523113521_1_thumb.jpg

joedirt says...

Another thought.. with a flash decompiler, these flashplayers DO bring up a still before you hit play, so maybe there is something in there. I'll have to investigate with a decompiler or wireshark.

joedirt says...

Uh... yeah, good luck with that:

Here is google's initial flashplayer loading the still screen before you hit play (identifying info replace with ......):
(I am HORRIFIED to know that google video is tracking my views attached to my gmail account.. so much for them not logging all that search history , etc.)

GET /googleplayer.swf?docId=-7552112268951569689&hl=en HTTP/1.1
Host: video.google.com
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
[[[Usual browser http header]]]
Cookie: MPRF=..............; rememberme=false; PREF=ID=...
GoogleVideoAnnotations=...; VPREF=...; GoogleVideoHomepageSettings=...; GoogleVideoPlaylistSettings=00d; TZ=240; GMAIL_RTT=203; GMAIL_LOGIN=........./......../........; S=gmail=........:gmail_yj=........:gmproxy=......:gmproxy_yj=.....:gmproxy_yj_sub=.....; SSDATA-DOMAIN=; GenrePageState=.......; sloc=en; GoogleVideoPlaylist=-......%2C-...........%2C........%2C-......; GoogleVideoPlaylistIndex=-1; GoogleVideoPlaylistParameters=1; SID=...........; NID=5=............

HTTP/1.1 200 OK
Last-Modified: Fri, 10 Aug 2007 19:56:30 GMT
Content-Type: application/x-shockwave-flash
Content-Length: 108010
Cache-control: private
Date: Thu, 30 Aug 2007 05:59:26 GMT
Server: GFE/1.3

GET /ThumbnailServer2?app=vss&contentid=fc62d88d323c031d&offsetms=125000&itag=w320&lang=en&sigh=X9LF3527M81H36GrIhUWj4KC8bM HTTP/1.1
Host: video.google.com
[[[[Usual browser http header]]]]
Cookie: [[[[same as above]]]]

HTTP/1.1 200 OK
Content-Type: image/jpeg
Cache-Control: max-age=604800
Server: btfe
Content-Length: 20721
Date: Thu, 30 Aug 2007 05:59:27 GMT

lucky760 says...

Let me give you an example of how we have been able to grab thumbnails thus far. If you browse to any LiveLeak video's page and view its HTML source, you'll find the thumbnail URL in a <link> tag in the header. On the page for a Google video, there is a thumbnail displayed in the top, left corner.

So for many of them it's a real matter of hacking through anything related to a specific video (including links, embed URLs, etc.) to locate where their thumbnails are stored. I imagine if you were to load an embed for any given host, the embed would need to load the thumbnail into the not-yet-playing player just to display it. If you were inclined and knowledgable enough to watch your computer's network traffic, you might be able to see where the embed's thumbnail is being downloaded from.

We're considering adding the ability for users to upload a screenshot of the video manually. In other words, screen capture while the video is playing, crop/save the image, then upload it on the video edit/submit page.

twiddles says...

I guess some of those other sources have been found since MetaCafe.com worked when I did findthumb, but I'll offer this for break.com if you need it:

Where src=http://embed.break.com/MzU4ODU3

GET /Embed/Redirector.aspx?Contentid=MzU4ODU3 HTTP/1.1

HTTP/1.1 302 Found
Date: Fri, 31 Aug 2007 01:14:31 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Location: http://media1.break.com/static/app/v1/global/swf/ghost_player_embed.swf?sVidLoc=http%3a%2f%2fmedia1.break.com%2fdnet%2fmedia%2f2007%2f8%2fchicago
-candy-factory-blown-up.flv&sThumbLoc=http%3a%2f%2fmedia1.break.com%2fdnet%2fmedia%2f2007%2f8%2fchicago-factory-blown-up.jpg&bEmbedded=1&showExpand=0&
contentURL=http%3a%2f%2fview.break.com%2f358857&sShareURL=http%3a%2f%2fview.break.com%2f358857%23TellAFriend&vo&bIsStatsActive=1&bIsReplayStatsActive=
1&volumeControl=20&sStatsUrl=http%3a%2f%2fstats.break.com%2finvoke.txt&debugFlag=1&iSiteID=1&iContentID=358857&autoplay=0&embed=2&contentidencoded=MzU
4ODU3
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Length: 806

<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="http://media1.break.com/static/app/v1/global/swf/ghost_player_embed.swf?
sVidLoc=http%3a%2f%2fmedia1.break.com%2fdnet
%2fmedia%2f2007%2f8%2fchicago-candy-factory-blown-up.flv&
sThumbLoc=http%3a%2f%2fmedia1.break.com%2fdnet%2fmedia%2f2007%2f8%2fchicago-factory-blown-up.jpg
&bEmbedded=1&showExpand=0&contentURL=http%3a%2f%2fview.break.com%2f358857%23TellAFriend&vo&bIsStatsActive=1&bIsReplayStatsActi
ve=1&volumeControl=20&sStatsUrl=http%3a%2f%2fstats.break.com%2finvoke.txt&debugFlag=1&iSiteID=1&iContentID=358857&autoplay=0&a
mp;embed=2&contentidencoded=MzU4ODU3">here</a>.</h2>
</body></html>

twiddles says...

VideoJug:

SRC=http://www.videojug.com/film/player?id=8a9e1653-3371-0a1c-92ad-ff0008c99cf8

GET /views/film/playlist.aspx?id=8a9e1653-3371-0a1c-92ad-ff0008c99cf8&items= HTTP/1.1


HTTP/1.1 200 OK
Date: Fri, 31 Aug 2007 01:38:03 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Cache-Control: public, max-age=5
Expires: Fri, 31 Aug 2007 01:38:09 GMT
Last-Modified: Fri, 31 Aug 2007 01:33:09 GMT
Vary: *
Content-Type: text/plain
Content-Length: 1663

<?xml version="1.0"?>
<Playlist PageUrl="http://www.videojug.com/film/how-to-avoid-trapped-arm-whilst-cuddling-in-bed" Title="How To Avoid Trapped Arm Whilst Cuddling In Bed" ImageUrl="http://content3.videojug.com/8a/8a9e1653-3371-0a1c-92ad-ff0008c99cf8/how-to-avoid-trapped-arm-whilst-cuddling.Player.jpg">
<PreRoll />
<PostRoll AdServer="http://adserver.adtech.de/addyn|3.0|671|1261088|0|0|ADTECH;key={0};grp={1};misc={2}" />
<Locations>
<Location Name="content.videojug.com" Url="http://content.videojug.com/8a/8a9e1653-3371-0a1c-92ad-ff0008c99cf8/" />
<Location Name="content3.videojug.com" Url="http://content3.videojug.com/8a/8a9e1653-3371-0a1c-92ad-ff0008c99cf8/" />
<Location Name="direct" Url="http://content.videojug.com/8a/8a9e1653-3371-0a1c-92ad-ff0008c99cf8/" />
</Locations>
<Slates />
<Items>
<Media Type="Video" Prefix="how-to-avoid-trapped-arm-whilst-cuddling" Title="How To Avoid Trapped Arm Whilst Cuddling In Bed" />
</Items>
<Credits />
<Shapes>
<Shape Code="PSP" Locations="content.videojug.com, direct" />
<Shape Code="3GP" Locations="content.videojug.com, direct" />
<Shape Code="FS7" Locations="content.videojug.com, direct" />
<Shape Code="FL7" Locations="content3.videojug.com, direct" />
<Shape Code="FS8" Locations="content.videojug.com, direct" />
<Shape Code="FL8" Locations="content3.videojug.com, direct" />
<Shape Code="WS9" Locations="content.videojug.com, direct" />
<Shape Code="WL9" Locations="content.videojug.com, direct" />
<Shape Code="POD" Locations="content.videojug.com, direct" />
</Shapes>
<Promotions />
</Playlist>

twiddles says...

Comedy Central:

<embed FlashVars='config=http://www.comedycentral.com/motherload/xml/data_synd.jhtml?vid=91009%26myspace=false'


<?xml version="1.0" encoding="iso-8859-1" ?>
<app graphics_library="...">
<module src="..." x="..." y="..."></module>
<module src="..." x="..." y="...">
<!-- START VIDEO PLAYER CONFIG -->
<boot interval="..." />
<player>
<src>...</src>
<buffer>...</buffer>
<timeout>...</timeout>
</player>
<lineup></lineup>
<emailer>
<requiredFields>...</requiredFields>
<submit>...</submit>
</emailer>
<rating>
<get>...</get>
<set>...</set>
</rating>
<embedMyspace>
<embed>
<expires></expires>
<content><![CDATA[<embed FlashVars='...' src='...' quality='...' bgcolor='...' width='...' height='...' name='...' align='...' allowScriptAccess='...' allownetworking='...' type='...' pluginspage='...r'></embed>
]]></content></embed>
</embedMyspace>
<embedOther>
<embed>
<expires><![CDATA[...]]></expires>
<content><![CDATA[<embed FlashVars='...' src='...' quality='...' bgcolor='...' width='...' height='...' name='...' align='middle' allowScriptAccess='...' allownetworking='...' type='...' pluginspage='...'></embed>
]]></content></embed>
</embedOther>
<targetDomain>...</targetDomain>
<isMyspace>...</isMyspace>
<isSyndicateable>...</isSyndicateable>
<playlist>
<element vidtype="1" parentid="" parenttype="" scrubstatus="true" playVideo="false">
<videoid>91009</videoid>
<syndicatable>...</syndicatable>
<uncensored>...</uncensored>
<videourl>
<![CDATA[http://www.comedycentral.com/motherload/xml/flv_xml_gen.jhtml?ml_video=91009&hiLoPref=lo]]>
</videourl>
<title><![CDATA[...]]></title>
<shortTitle><![CDATA[...]]></shortTitle>
<hpTitle><![CDATA[]]></hpTitle>
<description><![CDATA[...]]></description>
<airdate>...</airdate>
<startdate>...</startdate>
<views>...</views>
<duration>...</duration>
<rating>...3</rating>
<imageurl>http://www.comedycentral.com/images/shows/tds/videos/stewart/12100_jon_a_m4.jpg</imageurl>
<contentid></contentid>
<premierdate>...</premierdate>
<datetimeadded>...</datetimeadded>
<tuneIn>...</tuneIn>
<tuneInLink>...</tuneInLink>
<relatedshow>...</relatedshow>
<showtype>...</showtype>
<author>...</author>
</element>
</playlist>
</module>
</app>

lucky760 says...

Damn, twiddles. Nice work. Thanks a lot.

The Break one we had covered but the other two are excellent additions. Way to dig sift into those things. I'd applaud your contributions more if I could.

For the record, all the hosts for which we are currently able to find thumbnails are:


  • YouTube

  • Google

  • MetaCafe

  • iFilm

  • LiveLeak

  • Break


And soon, thanks to twiddles:

  • VideoJug

  • Comedy Central


So, outstanding at the moment:

  • MySpace

  • The Onion

  • DailyMotion

  • Revver

  • Vimeo

  • Yahoo!

  • vSocial

  • ForaTV


twiddles says...

Yahoo!

Potential way to get a thumb:
If vid=1017821 Then src="http://thmg01.video.search.yahoo.com/image/1017821_01"
From what I can tell it will always be thmg01 and end in _01 however there is a thmg02 registered.

<embed src='http://us.i1.yimg.com/cosmos.bcst.yahoo.com/player/media/swf/FLVVideoSolo.swf'
flashvars='id=3829686&emailUrl=http%3A%2F%2Fvideo.yahoo.com%2Futil%2Fmail%3Fei%3DUTF-8%26vid%3D1017821%26fr%3D%26cache%3D1&imU
rl=http%253A%252F%252Fvideo.yahoo.com%252Fvideo%252Fplay%253Fei%253DUTF-8%2526vid%253D1017821%2526cache%253D1&imTitle=Pimp%2BM
y%2BRon%2BPaul%2B%2528feat.%2BR.%2BKelly%2529&searchUrl=http://video.yahoo.com/search/video?p=&profileUrl=http://video.yahoo.c
om/video/profile?yid=&creatorValue=cXFxcXF0aGF0c3NpeHFz&vid=1017821' type='application/x-shockwave-flash' width='425' height='350'></embed>

twiddles says...

MySpace

<embed type="application/x-shockwave-flash" src="http://lads.myspace.com/videos/myspacetv_vplayer0005.swf" id="vplayer" name="vplayer" bgcolor="#000000" quality="high" allowscriptaccess="always" flashvars="m=2055862498&a=0&ap=1&v=2&culture=en-US" height="281" width="350">

query: http://mediaservices.myspace.com/services/rss.ashx?type=video&videoID=2055862498

response:

<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:media="http://search.yahoo.com/mrss" xmlns:myspace="http://myspace.com/">
<channel>
<title>...</title>
<description>...</description>
<link>...</link>
<lastBuildDate>...</lastBuildDate>
<docs>...</docs>
<generator>...</generator>
<myspace:friendID>...</myspace:friendID>
<item>
<title>...</title>
<pubDate>...</pubDate>
<media:thumbnail url="http://myspace-498.vo.llnwd.net/02055/89/42/2055862498_thumb0.jpg" />
<media:content url="httphttp://content.movies.myspace.com/0020558/89/42/2055862498.flv" type="video/x-flv" medium="video" status="Unknown" duration="127" />
<myspace:mediaID>...</myspace:mediaID>
<myspace:mediaSource>...</myspace:mediaSource>
<myspace:itemID>...</myspace:itemID>
<myspace:itemType>...</myspace:itemType>
<myspace:friendID>...</myspace:friendID>
<myspace:version>...</myspace:version>
<myspace:translatedStrings AddToProfile="..." By="..." Comment="..." MoreFromUser="..." PlayAgain="..." Ratings="..." RemoveFromProfile="..." Views="..." />
</item>
</channel>
</rss>

twiddles says...

DailyMotion

Where embed src="http://www.dailymotion.com/swf/1B00tXZ6vs7zik8CP"

GET /swf/1B00tXZ6vs7zik8CP HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)
Host: www.dailymotion.com
Connection: Keep-Alive


HTTP/1.0 302 Found
Expires: Sun, 02 Sep 2007 06:46:51 +0200
P3p: policyref="http://www.dailymotion.com/w3c/p3p.xml", CP="IDC DSP CURa ADMa DEVa TAIa OUR BUS IND UNI COM NAV INT"
Set-Cookie: [...]
X-Dm-Backnode: soft-14.srv.dailymotion.com:80
Date: Sat, 01 Sep 2007 04:46:51 GMT
Connection: close
Location: http://www.dailymotion.com/flash/flvplayer.swf?
rev=1187290581&statEnabled=1&selfURL=http%3A%2F%2Fwww.dailymotion
.com%2Fl0ckergn0me%2Fvideo%2Fx2uvhj_contact-list-management-with-
plaxo_tech&autoStart=0&logURL=http%3A%2F%2Flogger.dailymotion.com
%2Fvideo%2Faccess%2Fx2uvhj%3Fsession_id%3D%26referer%3D%26blog_key
%3D1B00tXZ6vs7zik8CP%26key%3D5kuhm7jqzb0jrgybkjnobyo&
previewURL=http%3A%2F%2Flimelight-719.static.dailymotion.com
%2Fdyn%2Fpreview%2F320x240%2F4799719.jpg

%3F20070829171536&url=http%3A%2F%2Fwww.dailymotion.com%2Fget
%2F14%2F320x240%2Fflv%2F4799719.flv%3Fkey%3Daa19a32f7c0a82dac
e9c76e6c450c9d512e4849&playerURL=http%3A%2F%2Fwww.dailymotion
.com%2Fvideo%2Fx2uvhj_contact-list-management-with-plaxo_tech
&playerURLTarget=_blank&allowZoom=2
Server: DMS/

9685 says...

Hi. Not sure if I'm missing something. Lucky760 indicates that the following thumbnail techniques have all been covered. Where? On this page, or am I missing a link to another page? I can see some solutions, but not all of these:

YouTube
Google
MetaCafe
iFilm
LiveLeak
Break
Comedy Central
MySpace
DailyMotion
VideoJug
Revver
Yahoo!
Vimeo

Are all of the solutions to these on this page??

thanks
David.

10820 says...

just figured out how to pull myspace thumbs then found you guys....typical.
has anyone actually written any scripts to embed these on any sites?

i did the myspace but it only works on windows or local PC not on unix server.
its a long process, about different 5 steps to get 1 thumb.. oh, but it is good to achieve it.

lucky760 says...

>> ^ricktee76:
just figured out how to pull myspace thumbs then found you guys....typical.
has anyone actually written any scripts to embed these on any sites?
i did the myspace but it only works on windows or local PC not on unix server.
its a long process, about different 5 steps to get 1 thumb.. oh, but it is good to achieve it.


Yep. We are using these scripts on this site! That's why we were trying to suss them out.

And they are all working on these servers which are running Linux. Of course, the methods are not fool-proof and some thumbs still cannot be gotten.

14525 says...

How about facebook? The url's for videos are like this:

http://www.facebook.com/video/video.php?v=630563174283

The video ID for the above is: 630563174283

The thumbnails look like this, but it's not as simple as substituting the videoID:
http://vthumb.ak.facebook.com/vthumb-ak-sf2p/v643/34/70/10200882/b10200882_630563174283_670.jpg

here's the thumbnail URL for another video:

http://vthumb.ak.facebook.com/vthumb-ak-sf2p/v643/34/70/10200882/b10200882_636853997423_2685.jpg

Their embed code looks like this, but I'd like to figure out the thumbnail url purely from the video id. Any ideas?

<embed type="application/x-shockwave-flash" src="http://b.static.ak.fbcdn.net/swf/mvp.swf?8:143195" width="756" height="425" style="" id="so_mvp_swf_498a8ae60124c6e03638827" name="so_mvp_swf_498a8ae60124c6e03638827" bgcolor="#000000" quality="high" allowScriptAccess="always" scale="showall" allowFullScreen="true" wmode="window" flashvars="video_src=http%3A%2F%2Fvideo.ll.facebook.com%2Fvideo-ll-sf2p%2Fv1182%2F151%2F66%2F636853997423_17867.mp4&stage_width=756&stage_height=425&m
otion_log=%2Fvideo%2Fmotion_log.php&video_id=636853997423&video_length=606770&video_seconds=606&video_category=0&video_rotation=0&video_href=http%3A%2
F%2Fwww.facebook.com%2Fvideo%2Fvideo.php%3Fv%3D636853997423&fbt_play_again=Play+Again&fbt_go_to_video=Go+to+Video&fbt_next_video=Next+Video&fbt_share=
Share&video_player_type=video_player_permalink&video_width=756&video_height=425&slate_src=http%3A%2F%2Fstatic.ak.fbcdn.net%2Fswf%2Fmvp_slate.swf%3F8%3
A134155&tail_slate_src=http%3A%2F%2Fstatic.ak.fbcdn.net%2Fswf%2Fmvp_tail_slate.swf%3F8%3A134155&next_video_url=http%3A%2F%2Fwww.facebook.com%2Fvideo%2
Fvideo.php%3Fv%3D630563174283&video_owner_name=Facebook+Platform&video_owner_href=http%3A%2F%2Fwww.facebook.com%2Fplatform&video_timestamp=Added+about
+a+month+ago&video_title=How+To%3AShare+to+Facebook+Feed+with+Facebook+Connect+in+10+Minutes&thumb_url=http%3A%2F%2Fvthumb.ak.facebook.com%2Fvthumb-ak
-sf2p%2Fv643%2F34%2F70%2F10200882%2Fb10200882_636853997423_2685.jpg&string_table=http://static.ak.fbcdn.net/js_strings.php/t85681/en_US&swf_id=so_mvp_
swf_498a8ae60124c6e03638827&video_autoplay=1"/>

lucky760 says...

That's actually a pretty easy one.

Step 1) Given the video's embed code, find the video id:
630563174283

Step 2) Load the HTML at the video's url with the id you found:
http://www.facebook.com/video/video.php?v=630563174283

Step 3) Look for the line with "thumb_url" on it (quotes inclusive):
("thumb_url", "http%3A%2F%2Fvthumb.ak.facebook.com%2Fvthumb-ak-sf2p%2Fv643%2F34%2F70%2F10200882%2Fb10200882_630563174283_670.jpg")

Step 4) Use a regular expression to extract and url decode the value in quotes after "thumb_url":
http://vthumb.ak.facebook.com/vthumb-ak-sf2p/v643/34/70/10200882/b10200882_630563174283_670.jpg

Done!

14525 says...

Lucky760, thanks! I've actually already tried doing exactly that. The only problem is that with the exception of Facebook's own instructional videos, the video pages aren't publicly accessible unless you're logged in. I'd have to resort to logging in first and then extracting the data from the page. My goal had been to see if someone could spot an obvious algorithm that was generating the data after http://vthumb.ak.facebook.com/vthumb-ak-sf2p in the thumbnail URL (besides the ID). I think I'm going to let our users paste in the embed code & parse the values from that instead of the page URL.

Paul

Send this Article to a Friend



Separate multiple emails with a comma (,); limit 5 recipients






Your email has been sent successfully!

Manage this Video in Your Playlists

New Blog Posts from All Members