June 24, 2001 10:17 AM

I am guessing that your vfat partition is a FAT32 partition. FAT32 has a 2GB file limit as well.
I am using W2K with NTFS. NTFS does not have this limit.

June 24, 2001 3:53 PM

Originally posted by Marc:
What you're describing is an application problem on the linux side, since you can get a file larger than 2 gig but your application fails to read it. If the file suddenly stopped writing at two gigs, like it does for your dos machine, that would be a kernel problem.

When I saw mplex on the TiVo do this across NFS to a Reiser file system (which shouldn't have a 2GB limit), I thought it was the old GLIBC on the TiVo.

So, I compiled "mplex" for a SuSE 7.1 system, and used dual netcats to transfer the data to the mplex on the PC. Mplex, again, stopped writing to stdout at 2GB (to a RiserFS)!

I'm at a loss.

I'm piping the output from mplex to "splt -b 650m - filename.mpeg" to keep file sizes down.

(The resulting files can be played via vlc with "cat filename.mpeg* | vlc -V sdl /dev/stdin".)

What, specifically, needs to be done to get the filesystem/kernel to allow files >2GB?

[ June 24, 2001: Message edited by: cworley ]

June 24, 2001 11:02 PM

Or use more bits! The 2GB limit comes from people using signed 32 bit integers as the file pointer... move it to unsigned and it moves to 4GB, use more bits and you can access larger files! Yes, doing larger math is slower since it has to be emulated if the processor doesn't support native types larger than 32 bits...

June 27, 2001 10:43 AM

Originally posted by dotorg:
Ugh this is really pissing me off.


Are there any options for multiplexing the audio/video and reencoding it to a slower bitrate under Linux that actually work with large files?


The most recent mplex won't touch files over 2 gig, even though most of the "system" utils are fine.

It's ugly too. The best info I've found is:
http://www.suse.de/~aj/linux_lfs.html

SuSE >= v7.0 should handle it (with their 2.2.x modified kernels, or 2.4 kernels). My problem might be due to my system being a 6.3...7.1 hybrid (When I need to upgrade, I upgrade to the latest, which usually has dependancies, like glibc that need to be upgraded, but I don't upgrade the entire system).

For example, if I'm using stdout, then my shell is responsible for handling the file size. For "open", you need "O_LARGEFILE" in your flags. For "lseek", you need _FILE_OFFSET_BITS defined for the compiler.

I doubt the TiVo kernel/libc can handle this at all. So, if a file is being written to an NFS mounted partition, the server may be able to handle 64 bit file inodes, but, I'm guessing, the old 2.1.24 kernels NFS client can't, and the mix will be unpredictable.

Streams should be okay. A netcat connecting the client to the server shouldn't go berserk when more than 2GB of data has flowed thru.

Some other useful links on the subject:
http://lists.svlug.org/pipermail/svlug/2000-September/003486.html
http://lists.omnipotent.net/reiserfs/200010/msg00031.html