I'll have to try this out as I have a bunch of v1.3 media that the standard ExtractStream 0.2 errors out with.
Btw.. do you have plans to submit this code to the original ExtractStream guys so they can integrate this into the mainline?
Thanks again..
--David
Hi all,
For anyone whose interested, I've written a little program to convert a tyStream extracted with ExtractStream -s into mpeg audio and video streams. It's based on PlayStream.c, and works at least with the tyStream from my HDR112 with Tivo 1.3.0.
I wrote it because I was having trouble with ExtractStream on my tivo: tons of "bogus sequence" errors. I haven't really looked into whether it'll work with the Tivo 2.0 software.
I've successfully muxed the audio and video streams back together using mplex, and I can play the mpeg2 program stream I get with no problem using mplayer.
You can get it from my tivo page at:
http://electron.mit.edu/~gsteele/tivo/
Cheers,
Gary.
Discussion: Add a Comment | Comments 1-15 of 15 | Latest Comment
View unverified member's comment - posted by Eric Bautsch
Gary this is absolutely brilliant.
One short question:
I can now get valid mpeg2 files from the tivo. I use ExtractStream-0.2, ConvertStream-0.1, and the mplex that comes with ExtractStream-0.2. When I play the stream however, I find the Audio and Video out of sync.
In fact after about 15mins it becomes unbearably out of sync, it gets worse as I skip through the stream.
Video is ahead of Audio.
Now I don't know where the problem is. It could be ExtractStream, ConvertStream, mplex, or even the playback software.
I'm running ConvertStream and mplex on Solaris 8 on x86, and finally play on WindozeNT using a demo version of PowerDVD (haven't got vlc working properly on either Solaris x86 or SPARC).
If anyone has experienced similar problems or even has a solution, I would very much like to hear from you. (I have looked on the forum and on google but haven't found anything relevant, which is not to say that I overlooked it.)
Thanks a lot.
Eric
View unverified member's comment - posted by Gary Steele
Hi Eric,
I'd be a bit careful about the player: implement robust audio/video sync in a player is difficult.
You may want to try mplayer http://mplayer.sourceforge.net. It's an excellent command line player, and I understand that it now runs on solaris 8 x86. The mpeg2 support seems pretty good. If you've got an x86 box, you could always install linux... :)
I have been having some troubles lately with the output from ConvertStream: the video data seems to be mildly corrupted. It segfaults mpeg2dec when I am testing the video stream, and there are occasionaly unrelated bad frames in the audio stream. These lead to fatal errors with mplex, though, not what you're having. I'll post back here if I figure out my problem.
Cheers,
Gary.
View unverified member's comment - posted by Eric Bautsch
Hi Gary.
Thanks for your help.
At the moment I'm having slight disagreements with mplayer as to what a PC running Solaris looks like (from what my programming skills allow me to deduce).
I shall try compiling it on SPARC Solaris, or indeed try to install Linux on a spare PC. Unfortunately both these things will have to be done at home (as opposed to while I'm bored at work), so I will take a bit longer to complete them.
I will work on it though.
Thanks again.
Eric
View unverified member's comment - posted by Marc
Originally posted by Gary Steele:
I have been having some troubles lately with the output from ConvertStream: the video data seems to be mildly corrupted. It segfaults mpeg2dec when I am testing the video stream, and there are occasionaly unrelated bad frames in the audio stream.
I've attempted to modify ConvertStream with some of the changes present in ExtractStream to work on 2.0 and I found the same problem. I know why it's happening, but no easy fix.
If you notice when watching ExtractStream run there are at least 2 harddrive locations per FSid, often more. I get the mpeg2dec segfault exactly at the point where it switches to the 2nd buffer.
Obviously some garbage exists at the start of each hard drive block. Unfortunately I haven't been able to determine a way to detect that from the raw file extracted. The problem is the same for multi FSid shows, but that can at least be avoided by extracting each FSid seperately and concatting the mpeg later (which isn't hard)
Perhaps the ultimate fix would be modifying Extractstream to seperate each HD buffer into it's own file, or even to add a control block so convert can remove the garbage itself.
Personally, I'm more interested in the extract to raw, then convert on a PC approach because my Tivo still stutters while extracting to audio/video files and in over 100 shows tested, I've yet to have a successful extraction anyways. Besides, I prefer watching TV on my TV, not my computer.
[ July 17, 2001: Message edited by: Marc ]
View unverified member's comment - posted by Eric Bautsch
Update on my progress and a cry for help (again!).
Following a hint from Doug and then reading about it on the forum ( http://www.ptvupgrade.com/cgi-bin/ultimatebb.cgi?ubb=get_topic&f=2&t=000097 ) I got the CVS version of ExtractStream.
This version is a big improvement for me. I can now extract from the TiVo straight to two files (*.m2v and *.m2a) without going via the raw stream (This didn't work for ExtractStream-0.2, the video stream wasn't valid).
If I use this method and then run mplex (from mjpegtools-1.4.1) with mplex -f 3 -m 2 I find the audio/video sync problem greatly reduced. Unfortunately not completely. I still have a couple of seconds sync problems.
If I use the new ExtractStream and extract a raw stream and then use Gary's ConverStream, the audio/video sync problem is back to the old greater difference.
I'd much rather extract the raw stream from the tivo and then do all the processing intensive stuff on my unix box, really.
I have also tried different players (on Windoze only: PowerDVD and EMpgPlay_130). The player does not make a difference.
I'm still struggling with MPlayer ( http://mplayer.sourceforge.net/ ) and vlc ( http://www.videolan.org/ ).
MPlayer compiles under Solaris SPARC and also runs. The video is half width though, and there is no audio.
VLC compiles and runs under Solaris SPARC, but the sound is a constant white noise, nothing else. It compiles under a quickly installed RedHat 7.0, too, but it doesn't run, but I haven't invested too much effort into this at the moment, as my first priority will be to extract the streams properly and without sync problems.
I think I have my problem down now pretty much to a precise location in the process.
The way I see it at the moment is this (and I'm not 100% on it):
* Raw stream extraction works.
* Raw stream to m2v and m2a appears to be buggy.
* conversion from m2v and m2a to a complete mpeg2 movie works.
If I'm right, what I'd like to find out is what changes were made to the raw -> m2[av] conversion routine in ExtractStream and can we drill that sync problem down further in there, and also: Gary, from looking at this, do you see an obvious problem/difference in your ConvertStream software (which I'd rather be using)?
I guess I'm handing this all over back to you guys/girls, as you can probably tell that my programming skills are not all that fabulous.
Anyone got any more ideas?
All help will be greatly appreciated.
Eric
P.S. Tomorrow (bored at work) I will take a look myself at the differences between ExtractStream-0.2 and the CVS version I downloaded today, but my hopes are pretty slim that I'll even understand the code... :-(
View unverified member's comment - posted by topset
Notes:
CVS has a "diff" option -- you can see differences between versions.
vlc has audio plugins -- try different plugins.
Have you tried the netmplex in CVS?
View unverified member's comment - posted by Marc
Originally posted by Eric Bautsch:
If I use this method and then run mplex (from mjpegtools-1.4.1) with mplex -f 3 -m 2 I find the audio/video sync problem greatly reduced. Unfortunately not completely. I still have a couple of seconds sync problems.
If I use the new ExtractStream and extract a raw stream and then use Gary's ConverStream, the audio/video sync problem is back to the old greater difference.
Play with the -O option to mplex. I've found that 100 ms is about right.
I have also tried different players (on Windoze only: PowerDVD and EMpgPlay_130). The player does not make a difference.
Odd, I've had files that I can't open in TMPGEnc (due, I believe, to the garbage I described above) but could still play in PowerDVD. In fact, -everything- plays in PowerDVD. Even with P-dvd's codec's I haven't been able to get Win MPlayer to play any MPEG2, although I haven't put alot of time or care into it.
View unverified member's comment - posted by Eric Bautsch
topset:
I haven't tried netmplex yet. I just got it compiled on Solaris X86 with the following modifications:
the final linking requires libs -lsocket and -lnsl
In net.cpp I added at line 17:
#ifndef INADDR_NONE
#define INADDR_NONE 0xffffffff
#endif
It would appear that netinet/in.h on Solaris does not define INADDR_NONE. I don't know how valid this approach is, as I would really need to know what Solaris returns when it doesn't find the address (this is how I read the source code with my limited understanding, the define is borrowed from http://www.nendai.nagoya-u.ac.jp/global/sys/HTML/S/netinet%20in.h.html - just a google search for netinit/in.h).
I will try running netmplex later tonight at home.
I will also take a look at what audio plugins are available in vlc (which actually compiled) on SPARC Solaris later tonight. I've only got my laptop on me here at work and that does not have sound support under Solaris.
I have had a look at the code changes in ExtractStream, but to be quite perfectly honest: It's way above my head. I'm not really going to be of a lot of use trying to figure out what fixed it for me.
The comment at the top of ExtractStream however does say that it now has 1.3 backwards compatibility, but is also sais it's not working properly....
My Tivo runs 1.5.2-100-023 - I'm in the UK.
Marc:
On the DVD player front: I didn't make myself clear. Both PowerDVD and EMpgPlay_130 play all the streams. However in both the audio/video sync problem is the same. Thus I'm not just having a problem with the player, the fault is actually in the stream (well, that's how I interpret it).
I had played with the -O option originally, but without much success. I hate to admit that this was due to my very own stupidity though (it helps to read the source code occationally, as I tried the -O on the mplex that came with the tivo as opposed to the mjpegtools mplex).
After that I forgot about the option again, stupid me.
However: ./mplex --help
[...]
-O num Specify offset of timestamps (video-audio) in mSec
[...]
Does that not imply that this is a constant value, i.e. the audio/video sync problem does not increase as you watch the file further?
Originally, when I used ExtractStream-0.2, audio and video got further out of sync as I continued watching the stream.
Now I have been thinking (when playing around with it yesterday) that perhaps there is a constant difference now that I am using the CVS version of ExtractStream, but I put it down to wishful thinking on my part.
I shall have to extract some two hour film tonight to definitely get a result on this one (One hour StarTreks are just not long enough to make a definite statement).
Thanks a lot guys for all your help. Looks like I've got a lot to try when I get home...
Eric
View unverified member's comment - posted by Eric Bautsch
Success (partially, but usably)!
Marc:
I played again with the -O option. 100 msec didn't seem to make a great deal of difference, so I tried 500 msec (a lot, I know) but that to my surprise gave me a very adequate result with a 1hr show. In the very beginning it would appear as though I'm just slightly out of sync now, the other way round, audio before video, but bearly noticable.
In the middle though I'm absolutely spot on. And in the end the video is again slightly ahead of audio.
The whole thing still moves through the show, but I definetly have a result that I can watch without going grazy with the sync problem.
I have only tried one programme so far, but will obviously continue.
Thanks again for your help. Great.
topset:
I tried netmplex, too. Compiled on Solaris x86 as above.
First I tried with the TiVo as a client, running netmplex with -l -o <file>, but that came straigt back with:
# /var/tmp/netmplex/netmplex -l -o papermoon-cvs-netmplex.mpeg
accept: Resource temporarily unavailable
#
I then tried the other way 'round with:
netmplex -a [TiVo's IP address] -o myfile.mpeg2
That worked. I got a stream from the tivo. Complete and all. But I still have the same sync problem that I had running mplex after letting ExtractStream extract to <file>.m2[av]. Note: This is better than when extracting raw and then running through ConvertStream (but I'd still prefer to use ConvertStream, as I would very much like to keep processing on the TiVo to a minimum).
After downloading the stream, I shut down the PC side of things (CTRL-c). This prompted the TiVo to reboot. I suppose I should have shut down the TiVo side first. Just don't know if this is expected behaviour?
I haven't had another look at vlc yet...
View unverified member's comment - posted by foundation
I've been having trouble extracting some of my stored videos notably 1 show recorded 1/31/01 and one show recorded 4/17/00. So I thought I'd give Convertstream a try. I'm running 2.0.1
It gives me the following error when I try to execute it
bash: ./ConvertStream: cannot execute binary file
Is the version in the tar file compiled for x86 or Tivo?
Somebody mentioned using ExtractStream with -s for raw mode. Do you pipe the output to a file and then run convertstream on that file or do you run convertstream directly on the MFS media partition?
View unverified member's comment - posted by Marc
Originally posted by foundation:
Is the version in the tar file compiled for x86 or Tivo?
Linux x86.
Somebody mentioned using ExtractStream with -s for raw mode. Do you pipe the output to a file and then run convertstream on that file or do you run convertstream directly on the MFS media partition?
First option: ExtractStream -s to a file, then run ConvertStream on that file
View unverified member's comment - posted by daler
Originally posted by Gary Steele:
Hi all,
For anyone whose interested, I've written a little program to convert a tyStream extracted with ExtractStream -s into mpeg audio and video streams. It's based on PlayStream.c, and works at least with the tyStream from my HDR112 with Tivo 1.3.0.
I wrote it because I was having trouble with ExtractStream on my tivo: tons of "bogus sequence" errors. I haven't really looked into whether it'll work with the Tivo 2.0 software.
Cheers,
Gary.
Has anyone had any luck with this an 2.01? I've ported both ConvertStream and Mplex (from the ExtractStream archives) over to win32 and am working on a netmplex type program. I simply can't get ConvertStream to like the tystreams I pull off, though. Its always giving a checksum error on the packet. :(
AFAIK, the tystream IS good and I'm using the latest ExtractStream from CVS. I've tried both the pre-compiled ConvertStream and re-compiled on both Linux and Win32 and none like the checksum. :(
Before I go diffing source to ExtractStream CVS and ConvertStream, thought I would see if anyone has done that yet or knows whats going on with CS and 2.01?
Dale
View unverified member's comment - posted by Marc
Originally posted by daler:
Has anyone had any luck with this an 2.01?
I've been updating convertstream to work with 2.01. It is still imperfect.
The problem you're having is because the checksum methodology changed between 1.3 and 2.01. Currently I just have it commented out (much like Extractstream 0.2) and haven't updated it tot he new method (which ES 0.3 does)
Output is not mpeg2dec-clean, meaning mpeg2dec will segfault midstream while decoding. The output I get from Extractstream isn't either. In fact, I can't even open the stream (before or after mplex'ing) into TMPGenc. PowerDVD will play it.
It's close, I just wish I knew more about the formatting of MPEG streams. A bit more validation during the conversion would dramatically clean up it's output.
For now, I'll have the code up here.
[ July 29, 2001: Message edited by: Marc ]
View unverified member's comment - posted by daler
Originally posted by Marc:
I've been updating convertstream to work with 2.01. It is still imperfect.
The problem you're having is because the checksum methodology changed between 1.3 and 2.01. Currently I just have it commented out (much like Extractstream 0.2) and haven't updated it tot he new method (which ES 0.3 does)
Output is not mpeg2dec-clean, meaning mpeg2dec will segfault midstream while decoding. The output I get from Extractstream isn't either. In fact, I can't even open the stream (before or after mplex'ing) into TMPGenc. PowerDVD will play it.
It's close, I just wish I knew more about the formatting of MPEG streams. A bit more validation during the conversion would dramatically clean up it's output.
[ July 29, 2001: Message edited by: Marc ]
If I give you a win32 diff, can you fold the changes into yours? There isn't a large number of changes, it just helps me from having to re-port everything every time. I'v e ported mplex, netmplex and convertstream, so its not a big deal. How dows mplex handle the results? My main goal is to combine convertstream and mplex into a single program for easy extraction (one netcat).
Discussion: Add a Comment | Back to Top | Comments 1-15 of 15 | Latest Comment
Add Your Reply
Sponsored Links
Copyright © 2013
Capable Networks LLC.. All Rights Reserved.
RSS |
Privacy Policy |
Advertise With Us |
Terms of Use |
About Us |
Contact Us |
Site Map
Our Capable Community Network:
DVR playground |
Explore3DTV |
MediaSmart Home |
My Open Router |
DijitCommunity |
Robo Community |
Sansa Community |
TechLore |
TiVo Community
