从视频流下载内容视频,路径为。TS 或。M3u8文件通过实际代码,所以我可以使铬扩展

大多数网站上的视频都采用渐进式下载,这意味着视频可以下载到我的电脑上,而且很容易追踪。有很多扩展可以做到这一点,甚至在 dev-tools 中也很容易做到。

在某些网站上,视频是流媒体播放的。这意味着我们不仅要下载一个文件,还要下载大量的小软件包。在 dev-tools 中可以跟踪这些包。我感兴趣的网站是: http://www.rtlxl.nl/#!/goede-tijden-slechte-tijden-10821/c8e2bff7-5a5c-45cb-be2b-4b3b3e866ffb

包裹有一个.TS 分机。

- 复制请求的网址可以保存软件包

我不能播放这些文件。

我一定是做错了什么,或者我错过了什么。我想知道我哪里做错了。我想创建一个个人使用的铬扩展,它捕捉所有的软件包的网址。当我有了所有的 url 后,我想把它们传递给一个 php 脚本,这个脚本会下载它们并使用 ffmpeg 将它们粘贴到 mp4文件中。

请指导我如何下载这些文件的内容。

394099 次浏览

You would need to download all of the transport stream (.ts) files, and concatenate them into a single mpeg for playback. Transport streams such as this have associated playlist files (.m3u8) that list all of the .ts files that you need to download and concatenate. If available, there may be a secondary .m3u8 playlist that will separately list subtitle steam files (.vtt).

Copy and paste one of the .ts video files into a new tab in Chrome. Remove the identifying number of the .ts file (0,1,2,3 etc. or whatever number it is) and change the extension from ".ts" to ".mp4". That should bring up the video file in your browser as usual.

---> Open Firefox

---> open page the video

---> Play Video

Click ---> Open menu

Click ---> open web developer tools

Click ---> Developer Toolbar

Click ---> Network

---> Go to Filter URLs ---> Write "M3u8" --> for Find "m3u8"

---> Copy URL ".m3u8"

========================

Now Download software "m3u8x" ----> https://tajaribsoft-en.blogspot.com/2016/06/m3u8x.html#downloadx12

---> open software "m3u8x"

---> paste URL "m3u8"

---> chose option "One...One"

---> Click Download

---> Start Download

========================

image "Open menu" ===>

a busy cat

image "Developer Toolbar" ===>

a busy cat

image "m3u8x" ===>

enter image description here

enter image description here

using this post

  • Open Firefox / chrome

  • open page the video

  • Play Video

  • click F12 on keyboard -> network

  • in Filter URLs ts

  • copy link of ts

  • remove index and ts extension from link

    for example:

    http://vid.com/vod/mp4:vod/PRV/Yg0WGN_6.mp4/media_b180000_454.ts
    

    will be copied as

     http://vid.com/vod/mp4:vod/PRV/Yg0WGN_6.mp4/media_b180000
    

insert in below script under LINK

#!/bin/bash


# insert here urls
LINK=(


'http://vid.com/vod/mp4:vod/PRV/Yg0WGN_6.mp4/media_b180000' # replace this with your url


)


mkdir my-videos
cd mkdir my-videos


CNT=0


for URL in ${LINK[@]}
do
# create folder for streaming media
CNT=$((CNT + 1))
mkdir $CNT
cd $CNT


(


DIR="${URL##*/}"


# download all videos
wget $URL'_'{0..1200}.ts


# link videos
echo $DIR'_'{0..1200}.ts | tr " " "\n" > tslist
while read line; do cat $line >> $CNT.mp4; done < tslist


rm -rf media* tslist
) &
cd ..


done


wait

EDIT

adding script in python - runs on windows and linux

import urllib.request
import os
import shutil


my_lessons = [
#  http://vid.com/vod/mp4:vod/PRV/Yg0WGN_6.mp4/media_b180000_454.ts
"http://vid.com/vod/mp4:vod/PRV/Yg0WGN_6.mp4/media_b180000" # replace this with your url




]


lesson_dir = "my_vids"
try:
shutil.rmtree(lesson_dir)
except:
print "ok"


os.makedirs(lesson_dir)
os.chdir(lesson_dir)


for lesson, dwn_link in enumerate(my_lessons):
print ("downloading lesson  %d.. " % (lesson), dwn_link)
file_name = '%04d.mp4' % lesson
f = open(file_name, 'ab')
for x in range(0, 1200):
try:
rsp = urllib.request.urlopen(dwn_link + "_%04d.ts" % (x) )
except:
break
file_name = '%d.mp4' % lesson
print "downloading  %d.ts" % (x)
f.write(rsp.read())
f.close()






print "done good luck!! ==================  "

if the script fails, or downloads empty file, try removing the try wrap to see what fails

I made some changes to dina's answer to avoid attempting to download/combine 1200 parts if there aren't that many.

I also found it helpful to sort by waterfall in the network tab of chrome. This will sort by the time the files are downloaded, so when you are streaming a video the most recently downloaded parts will be at the top, making it easy to find the .ts links.

#!/bin/bash


# Name of the containing folder
GROUP="My Videos"


# Example link: https://vids.net/ABCAED/AADDCDE/m3u8/AADDCDE/AADDCDE_0.ts
# Insert below as: https://vids.net/ABCAED/AADDCDE/m3u8/AADDCDE/AADDCDE


# INSERT LINKS TO VIDEOS HERE
LINK=(
'Title for the video link'
'https://vids.net/ABCAED/AADDCDE/m3u8/AADDCDE/AADDCDE'
'Title for the next video'
'https://vids.net/EECEADFE/EECEADFE/m3u8/EECEADFE/EECEADFE'
)


# ------------------------------------------------------------------------------
mkdir "$GROUP"
cd "$GROUP"


I=0
while [ $I -lt ${#LINK[@]} ]
do
# create folder for streaming media
TITLE=${LINK[$I]}
mkdir "$TITLE"
cd "$TITLE"
mkdir 'parts'
cd 'parts'


J=$((I + 1))
URL=${LINK[$J]}


I=$((I + 2))


DIR="${URL##*/}"


# download all streaming media parts
VID=-1
while [ $? -eq 0 ];
do
VID=$((VID + 1))
wget $URL'_'$VID.ts
done


# combine parts
COUNTER=0
while [  $COUNTER -lt $VID ]; do
echo $DIR'_'$COUNTER.ts | tr " " "\n" >> tslist
let COUNTER=COUNTER+1
done
while read line; do cat $line >> $TITLE.ts; done < tslist


rm -rf tslist
mv "$TITLE.ts" "../$TITLE.ts"


cd ..
rm -rf 'parts'
cd ..


done

I needed to download HLS video and audio streams from a e-learning portal with session-protected content with application/mp2t MIME content type.

Manually copying all authentication headers into the downloading scripts would be too cumbersome.

But the task got much easier with help of Video DownloadHelper Firefox extension and it's Companion App. It allowed to download both m3u8 playlists with TS chunks lists and actual video and audio streams into mp4 files via a click of button while correctly preserving authentication headers.

The resulting separate video and audio files can be merged with ffmpeg:

ffmpeg -i video.mp4 -i audio.mp4 -acodec copy -vcodec copy video-and-audio.mp4

or with mp4box:

mp4box -add audio.mp4#audio video.mp4 -out video-and-audio.mp4

Tried Video DownloadHelper Chrome extension too, but it didn't work for me.

Addition to @aalhanane and @Micheal Espinola Jr

As m3u8x is only available for windows. Once you have identified the m3u8 url you can also use Jdownloader2 or VLC Media Player to download and concatenate the stream.

Jdownloader2: Just copy the m3u8 url when it the Jdownloader is open. It will recognize the stream in Linkgrabber tab.

VLC 3:

Open Network -> Paste m3u8 url -> Checkmark Streamoutput -> Select Settings. Choose output file, container , video and audio encoding. (e.g output.mp4, container: mpeg4, video: h264, audio: mp4a) Start Stream. It will not play the video, but encode it, showing the encoding progress by moving the video play back progress bar.

WARNING: Previously suggesteed chrome extension Stream Video Downloader contains malware. See reddit post

Easy youtube-dl example on macOS (in the command line Terminal; Windows supported too):

# List variants (resolutions/bitrates)
$ youtube-dl -F https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8
[generic] f08e80da-bf1d-4e3d-8899-f0f6155f6efa: Requesting header
[generic] f08e80da-bf1d-4e3d-8899-f0f6155f6efa: Downloading m3u8 information
[info] Available formats for f08e80da-bf1d-4e3d-8899-f0f6155f6efa:
format code           extension  resolution note
audio-English_stereo  mp4        audio only [en]
628                   mp4        320x180     628k , avc1.42c00d, video only
928                   mp4        480x270     928k , avc1.42c00d, video only
1728                  mp4        640x360    1728k , avc1.42c00d, video only
2528                  mp4        960x540    2528k , avc1.42c00d, video only
4928                  mp4        1280x720   4928k , avc1.42c00d, video only
9728                  mp4        1920x1080  9728k , avc1.42c00d, video only (best)


# Choose a variant to download, and use its format code below
$ youtube-dl --format 628 https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8
...
frame= 5257 fps=193 q=-1.0 Lsize=    6746kB time=00:03:30.16 bitrate= 263.0kbits/s speed=7.73x
video:6679kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.998669%
[ffmpeg] Downloaded 6907810 bytes
[download] 100% of 6.59MiB in 00:29


$ open f08e80da-bf1d-4e3d-8899-f0f6155f6efa-f08e80da-bf1d-4e3d-8899-f0f6155f6efa.mp4

Use the browser's Developer Tools > Network to get the m3u8 (HLS manifest) URL when starting a streaming video.

1) Please read instructions by @aalhanane (after "paste URL m3u8" step you have to type name for the file, eg "video" then click on "hand" icon next to "quality" and only after that you should select "one on one" and "download").

2) The stream splits video and audio, so you need to download them separately and then use the same m3u8x to join them https://youtu.be/he-tDNiVl2M (optionally convert to mp4).

3) m3u8x can download video without any issues but in my case it cannot extract audio links. So I simply downloaded the *.m3u8 file and searched for line which contains GROUP-ID="audio-0" and then scroll right and copied the link (!including token!) and paste it straight into "Quality URL" field of m3u8x app. Then "one on one" and download it similar to video stream.

Once I had both video and audio, I joined and success =)

p.s. in case automatic extraction will stop working in the future, you can use the same method to extract video links manually.

  • Get one Link from Network tab of developer tools
  • Remove index and ts extension from link

With following script you can save movie to Videos folder

Example usage:

download-video.sh https://url.com/video.mp4 video-name

download-video.sh

#!/bin/bash
LINK=$1
NAME=$2


START=0
END=2000


help()
{
echo "download-video.sh <url> <output-name>"
echo "<url>: x.mp4 (without .ts)"
echo "<output-name>: x (without .mp4)"
}


create_folders()
{
# create folder for streaming media
cd ~/Videos
mkdir download-videos
cd download-videos
}


print_variables()
{
echo "Execute Download with following parameters"
echo "Link $LINK"
echo "Name $NAME"
}


check_video()
{
i=$START
while [[ $i -le $END ]]
do
URL=$LINK'-'$i.ts
STATUS_CODE=$(curl -o /dev/null --silent --head --write-out '%{http_code}\n' $URL)
if [ "$STATUS_CODE" == "200" ]; then
break
fi
((i = i + 1))
done


if [ "$STATUS_CODE" == "200" ]; then
START=$i
echo "START is $START"
else
echo "File not found"
fi
}




download_video()
{
i=$START
e=$END
while [[ $i -le $END ]]
do
URL=$LINK'-'$i.ts
STATUS_CODE=$(curl -o /dev/null --silent --head --write-out '%{http_code}\n' $URL)
if [ "$STATUS_CODE" != "200" ]; then
break
fi
wget $URL
e=$i
((i = i + 1))
done


END=$e
}


concat_videos()
{
DIR="${LINK##*/}"


i=$START
echo "i is $i"
while [[ $i -le $END ]]
do
FILE=$DIR'-'$i.ts
echo $FILE | tr " " "\n" >> tslist
((i = i + 1))
done
while read line;
do
echo "gugu"$line
cat $line >> $NAME.mp4;
done < tslist


rm *.ts tslist
}


if [ "$1" == "" ]; then
echo "No video url provided"
help
else
LINK=$1
if [ "$2" == "" ]; then
echo "No video output-name provided"
help
else
NAME=$2
create_folders
print_variables
check_video
download_video
concat_videos
fi
fi

You can use Xtreme Download Manager(XDM) software for this. This software can download from any site in this format. Even this software can change the ts file format. You only need to change the format when downloading.

like:https://www.videohelp.com/software/Xtreme-Download-Manager-

While this shouldn't have ever been asked on SO and got through the vetting processing in the first place, I have no idea... but I'm giving my answer anyway.

After exploring basically all of the options presented here, it turns out the simplest is often the best.

First download ffmpeg from: https://evermeet.cx/ffmpeg/

Next, after you have got your .m3u8 playlist file (most probably from the webpage source or network traffic), run this command:

ffmpeg -i "http://host/folder/file.m3u8" -bsf:a aac_adtstoasc -vcodec copy -c copy -crf 50 file.mp4

I tried running it from a locally saved m4u8 file, and it didn't work, because the ffmpeg download procedure downloads the chunks which are relative to the URL, so make sure you use the website url.

I came up with an efficient parralelized one-line that concatenate a sequence of .ts files into one .mp4 file using GNU Parallel:

 parallel -k curl https://example.com/video/seg-{}-f4-v1-a1.ts ::: {1..279} >> result.mp4

The key is to replace the variant part of your url with {} and to set {a..b} with the lower and upper bound. That's it!

  • Download VLC Player
  • Media
  • Convert/Save
  • Network (Tab)
  • Enter URL of [playlist].m3u8
  • Follow remaining wizard steps to set the stream destination (File)
  • Set appropriate transcoding profile (MP4 at the time of this answer)
  • Watch video

As simple as these two commands:

wget https://example.com/videos/tschunks_{0..10}.ts

cat tschunks_{0..10}.ts > video.mp4

*- where "https://example.com/videos/tschunks_{0..10}"-part can be found in m3u8 file

Many websites package their videos as multiple Transport Stream files (.ts). If you wanted to output the full length video you would need to download all .ts file pieces and assemble them. The assembling can easily be done using the FFmpeg command line tool.

In case anyone is looking for a Java way of downloading TS files from video stream here's a good utility example and article that also goes into detail on how TS and M3U8 files work >> https://jet-cabral.medium.com/java-ts-video-downloader-a0fcf23ab84a

Once all .ts files are downloaded the ffmpeg command to assemble/concatenate the videos looks like so:

ffmpeg -i "concat:C:\users\your-name\file_0.ts|C:\users\your-name\file_1.ts|C:\users\your-name\file_2.ts|C:\users\your-name\file_3.ts

If you already know the url of the .m3u8 file that contains the .ts files you want to download. Then the following command downloads all the .ts files and outputs a single .mp4 file

ffmpeg -i "http://www.some-website.com/video/someM3U8_file.m3u8" -c copy -bsf:a aac_adtstoasc "full_video.mp4"