How to live transcode and stream HDV to MP4 using VLC and Linux

I’ve been trying to figure out a way to do this on the cheap for a long time and I finally figured it out today. This process allows you to grab HDV from a HDV Camera via firewire, feed it into linux, transcode the 25Mb/s mpeg-ts stream to a 4 Mb/s mpeg4 stream (inside a TS). This mpeg4 stream in turn can be viewed at full resolution (1920×1080) on a remote client running just vlc. Here is the prerequisites:

  1. A decent machine with working Firewire port (anything from the past 2-3 years should do). Laptops might work as well although I have not tried it yet. My machine is a Athlon 4200+ w/ 2GB of RAM and a 512 MB NVIDIA 7900.
  2. Ubuntu 7.10 (Gutsy Gibbon) installation CD.
  3. 4-pin to 6-pin Firewire cable.
  4. HDV Camera with Firewire out (I use a Canon HV20).
Okay so here we go, follow the steps below to get setup:
  1. Get Ubuntu 7.10 installed on your machine. This should be standard installation from the Live-CD.
  2. Reboot, do all adjustments to your display, get network setup, install all the updates.
  3. Using synaptic package manager install the following extra packages: ubuntu-extras, ffmpeg, dvgrab 3 and VLC plus anything else you might want.
  4. Connect the firewire camera to the computer and check /var/log/messages to make sure it gets recognized.
Now that you have the chain setup, it’s time to do a quick test and see if the system is working. Issue the following command from a xterm, making sure that the camera is turned on and in “Camera” mode.

sudo dvgrab -f hdv -noavc -nostop -|vlc -
You have to use sudo under ubuntu to get proper access to the firewire device. The above command runs dvgrab with hdv format and makes sure that 1394 AV/Device control is turned off (this way you can be in Camera mode and get a live feed). The nostop switch prevents dvgrab from sending stop commands to the camera everytime you stop it via Ctrl-C, which I though was a good thing. The last dash forces dvgrab to output to stdout, which we’ll then pipe into vlc (the dash for vlc tells it to use stdin as input).

If this works you should get a vlc window and be able to see live video from your HDV camera. If you didn’t then stop here and make sure you get this working first.
So now that we have dvgrab working, lets grab that 25Mb/s HDV stream and squish it down to 4Mb/s mpeg4 stream using the following command:

sudo dvgrab -f hdv -noavc -nostop -|vlc - --sout '#transcode{vcodec=mp4v,vb=4096,acodec=mpga,ab=256,scale=0.5,deinterlace,width=1920,height=1080}:duplicate{dst=std{access=udp,mux=ts,dst=receiver_ip_address:1234}}'
This is one massive command, the first part we already discussed, so lets take a look at the second half:

  • –sout is our output chain.
  • #transcode tells vlc that we first want to transcode the input using parameters to follow
  • {} contains our transcoding parameters
  • vcodec=mp4v sets the video codec to mpeg4 video
  • vb=4096 sets the bitrate of the transcoded video (4Mb/s)
  • acodec=mpga sets the audio codec to mpeg audio (mp3)
  • ab=256 sets the bitrate of the transcoded audio (256 Kb/s)
  • scale=1 sets the scaling value
  • deinterlace sets guess what?
  • width=1920 sets the width of the transcoded video to 1920 pixels
  • height=1080 sets the height of the transcoded video to 1080 pixels
  • :duplicate tells VLC that we want to duplicate the transcoded signal and send a copy of it to our receiver machine.
  • dst is the destination string
  • access=udp specifies UDP protocol
  • mux=ts sets multiplexing to mpeg-2 Transport stream
  • dst=receiver_ip_address:1234 is the ip address and port number of the receiving machine
So now you should be able to open up vlc on the receiver machine, goto File/Open Network menu and select UDP/RTP and specify port number 1234. Once you press OK, you should see the video stream on your receiver machine. Audio works as well and is perfectly synced since it’s captured by the HDV camera at the source and travels together with the video at all time. The delay is about 3 seconds.
This is a great way to quickly setup a HD Video Conference between a couple of locations. You could even modify the network portion of the chain to let VLC multicast the HD stream onto your network…..lots of possibilities. Enjoy :-)

Many Ayromlou

, , , ,

19 responses to “How to live transcode and stream HDV to MP4 using VLC and Linux”

  3. Hi Anonymous,

    Simply saying it does not work does not help :-). I’ve tried this on numerous occassions and have even cut and pasted the command off the blog page.

    You might be getting confused at the ‘sudo’ portion of this command. When you issue the command initially, the sudo is waiting for your password, and once it is verified it will run vlc.

    So try typing in your password next time after you issue the command.

    TTYL
    Many

  4. Nice tip! Haven’t tried yet with a HD camcorder, but I’ve been playing with vlc streaming recently (using a TV capture card).

    How much of CPU does it use when encoding a 1080i MPEG2 stream into a MPEG4?

  5. how do you save the stream on the 2nd PC while watching? i guess there is a setting in VLC. i’ll check it out.

  6. this is a very useful post, but i’d like to know a few more things …

    i want the whole command line sintax for the vlc, where can i get it?, I all ready know about “–longhelp –advanced” but it’s a little confusing for a newbie like me, i think that there is information missing.

    for example, what should i do if i wanted to convert from ogg to mp3?

    i don’t think i’m able to do that with your post and the vlc command line options, where can i get the information that i’m missing?

    thanx!!

  8. There is good (but a little outdated) documentation at http://www.videolan.org/doc/videolan-howto/en/.

    Thanks for blogging this – I hadnt considered live firewire capture until I came across this entry.

    Can I ask what version of VLC you are using? I ask because I am hitting a problem where my CPU (Q6600) is hitting 100% and my screen filling up with ” late buffer for mux input” messages.

    I am REALLY surprised that you were doing this on a 4200 and hoping to duplicate this.

  9. Hi Lee,

    Let’s just say I’m mighty surprised that you’re not able to do this with a Q6600……THAT’S QUADCORE!!!!……Hmmmm…..What camera are you using…..does it spit out HDV on it’s firewire port (or could it be an mp4 like stream….ie: AVCHD)?

    The reason I ask is that as the article mentions I’m doing this on a stok 4200 (dualcore). I’ve also since tried it on a small 2.2 Ghz C2D machine with 2 gigs of ram.

    Could it be the firewire controller maybe…..what chipset is the controller. I was using a PCI firewire board with a TI (Texas Instrument) chipset…..I’m wondering maybe the other chipsets drivers are a bit wonky or something….

    FYI. I was using either 0.8.6 or 0.8.5 at the time (don’t remember which exactly).

    Might also want to check your bios and see if there is a way to move the firewire to a different interrupt (some bioses let you do this still).

    One thing is for sure….Q6600 is MORE than enough to do this. The reason I say that is because I’ve been using one (with 4g ram) to do screen capture streaming @1920×1200 in a mp4 stream (it’s also on the blog….just search for vlc). The quadcore machine is the only one that can do that and not die miserably. If I try that on a Dual or a athlon I end up not being able to see the local mouse pointer since the CPU gets pegged…..My Q6600 though is smooth as butter.

    Try running the other scenario as well maybe (screen streaming at HD res). The instructions are (http://www.nerdlogger.com/2008/01/stream-your-linuxwindowsmac-desktop-as.html).

    Also have a look at the updated instructions for 8.04 (hardy) at (http://www.nerdlogger.com/2008/06/transcode-and-stream-dv-to-mpeg4-via.html).

    TTYL
    Many

  10. Cheers Many, I had a look at the screen streaming and hardy posts – raw capture from firewire doesn’t seem to work at all (Im on 8.10 with a hv30 btw) and the screen-streaming also gives me the “late buffer for mux input” error (plus terribly corrupt video).

    I wonder if this is a regression between 8.6 and 9.x – I’ll have to try and get an older version to compile see if that gets rid of the problem :(

  11. Hi Lee,

    Make sure the dvgrab line is not waiting for a password for sudo. That might be the issue. I will try it on a 8.10 machine and see if I can get it going and report back…..maybe tomorrow or next day.

    Let me know if you figure it out.

    TTYL
    Many

  12. Thanks Many, I had a chance to play about with the screen-streaming on a different machine to the quad and found I wasn’t getting the mux error on that, so I am assuming there is an issue with my machine/VLC installation at work. Will let you know how I get on with this since its far to much fun to concede failure on ;)

  13. Hi Lee,

    Okay that sounds more like it. I hate to say it, but something sounds really wrong with the Q6600 machine. I’m still thinking interrupt and/or memory issues maybe.

    Also try uninstalling VLC and reinstall it and make sure you check the wipe old configuration during install. Maybe you’d configured something that’s clashing with this stuff.

    TTYL
    Many

  14. Hi Many, sorry to trouble you again but I’ve a few more questions, namely what sort of network bandwidth do you get while streaming, what sort of CPU usage and do you see motion artifacts on the other end?

    I’m asking because I’ve been able to stream successfully when using VLC but the only way I’ve been able to stream with decent quality (no motion artifacts) and low CPU usage was with the by-chance usage of :

    sudo dvgrab -f hdv -noavc -nostop -| vlc – –sout ‘#transcode{vcodec=,vb=4096,acodec=mpga,ab=256,scale=1,deinterlace,width=1920,height=1080}:duplicate{dst=std{access=udp,mux=ts,dst=192.168.11.3:1234}}’

    Now if I understand whats going on here VLC isnt actually transcoding at all (I get minimal CPU usage – exactly what I need) but is just pumping the firewire stream down the line. BUT on the receiving end VLC sees this as h264 video (locally VLC sees it as mpgv)! My network usage is through the roof at 3MiB though.

    As said though the video quality is as good as I get locally so I’m somewhat confused now…more or less a normal state for me :)

  15. Hi Lee,

    Couple of things…..mp4v is mpeg 4 video which is the video component of a mpeg 4 stream. mpeg can be version 1,2 or 4. In our case we’re using 4 which is also known as H264 (they are cousins so to speak). Do a wikipedia search for h264 if you want to know the details.

    Now….let me guess…..you used the gnome-system-monitor gizmo on your linux box (the video encoder) over a remote X window link….right? That’s why you’re getting 3MiB as your network traffic contains the 264 stream and also the remote desktop application. I’m running the application under hardy-heron right now and on the system monitor on the encoder I read 350-500 KiB per sec. which is correct (roughly 4Mb/s depending on motion in the video).

    If the above is not the case then there must be something else messing up your system. Please check the numbers again. I’m sure you’re missing something.

    Let me know either way…..

    TTYL
    Many

  16. Hi again Lee,

    Forgot about the CPU usage…..I’m on a Core2Duo T7700 right now 2.4Ghz. I see one of the cores getting hammered (100%), when there is motion in the video, in a 2 second cycle. I think the linux scheduler moves the encoding thread around from one core to the other. The other core is about 10-20%.

    As far as motion artifacts….yes I see them when I move around in the video alot. Afterall you’re squeezing 25Mb/s into 4Mb/s.

    TTYL
    Many

  17. Again Many…many thanks ;)

    I am starting to get somewhere with this and knowing that you are seeing motion artifacts is a huge help forward since we’re working on VERY high quality recordings….and several of them (4) simultaneously. Oh and to have a DVD + iPod compatible copy/version(s) available directly afterwards. You’ll be able can smell the smoke from Canada ;)

    Anyways it looks like my next step is going to be to look at the Hauppage HD PVR with a view to getting a compressed stream out of that and using something like an atom based machine to relay it and then encoding/re-encoding on a core I7 based machine (my quad core isnt *quite* up to the task of re-encoding a stream to DVD). All interesting stuff.

    Again many thanks for your time and patience,
    regards
    Lee

  18. I used to be able to do this, but since i’ve upgraded my box to Ubuntu 8.10 i can’t anymore.

    Seems that the problem is that vlc 0.9.4 is somewhat different from 0.8.x versions.

    I’m not even able to do the easy part, namely

    dvgrab -f hdv -noavc -nostop -|vlc –

    This outputs

    libdvdnav: Using dvdnav version 4.1.2 from http://dvd.sf.net
    libdvdnav: vm: failed to open/read the DVD
    [00000417] access_directory access error: -: The file or directory does not exist

    “stdout”: buffer underrun near: timecode ??:??:??.?? date ????.??.?? ??:??:??
    This error means that the frames could not be written fast enough.

    It must be some changes to the commandline options in vlc.

  19. I found the workaround for VLC 0.9.4 at least. Use

    dvgrab -f hdv -noavc -nostop -|vlc /dev/stdin

    instead.

Leave a Reply