Digital ATV Project

Live Video with the New Serialiser

See this link for F1FAU's Update on his Hardware/Software for Live DATV, USB QPSK parts 1 & 2

---------------------------------------------------------------------------------------------------------

Software Configuration Update 11-1-2010

Live 2Ms/s DATV now working well with Linux and the Hauppauge PVR cards

3Ms/s tested as working with 24MHz xtal, 4Ms/s not possible with current Serialiser chip - 1-5-2010

There may be a few errors until i get some feedback... please email me if you find any!

Click here for Live DVD image Information...

Recent work into making the Poor Man's DATV Transmitter project better have led to quite a few improvements!
Firstly due to the work of Christian F1FAU for his idea of using the hauppauge PVR-150 card to generate the MPEG-2 stream in hardware and sencondly because i have had more time to test the possibilities!

The advancements have been possible using Linux OS which has more software utilities available for MPEG-2 streams and also one for DVB transport stream generation, that's why we will using Linux to generate a better DVB compliant stream than before.

The OpenCaster software suite (originally called JustDVB-IT) is where the main advancement has come from, this package let's us mux many transport streams together and to generate the service information that needs to be sent with the transport stream to make it DVB compliant. The package is not that straightforward to use and has taken me a few months to get to the point where i have a usable transport stream, hopefully i can explain what i did to make it easier for you!

All of the infomration below is based on Ubuntu 8.04 version of Linux, it all works fine now with ubuntu 9.10 too.

Here is a NEW!! layout of the data flow for 2Ms/s configuration...

Ubuntu Live DVD image available 21st Jan 2010

Now there's no need for you to install linux on your PC, run it direct from the DVD!

I have now created the first bootable DVD disk image of my Ubuntu 8.04 system that you will be able to run on your pc with all of the software below pre-installed. The image will run straight from a DVD in your pc without any modification to your windows system, all be it a little slow to start it gives you the chance to test my DATV system without even installing Linux on your pc!

The 64bit and 32bit versions are now available. The images will be split into 4 or 5 files as free online storage that allows 850Mb in a single file does not seem to exist yet they offer 10GB of space!

It will be in .rar format so you will need WinRAR or similar to extract the parts into one disk image. You can put the image on a DVD or probably a USB drive but that's up to you to figure out, i haven't got a big enough USB drive to test it!

You can get the disk images here: http://www.4shared.com/dir/28791122/84ef05be/sharing.html

Or here using dropbox.com server...

Ubuntu 8.04 64bit version parts: 1 2 3 4 5

Ubuntu 9.10 32bit version parts: 1 2 3 4

Your DVD authoring software should be able to directly write the image to a DVD. Once written you should be able to boot straight from the DVD when you start your computer, press Enter when prompted with the menu.

Reading the infomation below may become useful when you get more familiar with how it works.

System Requirements..

You will need a pc with >2GHz processor to run 1000 symbol rate and >2GHz DUAL CORE system to run 2000 symbol rate.

Good luck!

How to Install the software manually..

The layout of all the folders where you can install the various programs will be something like the details below, you will have a different username

Remember the folder and file names are case sensitive in linux.

In linux the root filesystem is "/" this is the same as "C:\" in windows. The home folder is just like 'documents and settings' folder and so on... It's best to use your home folder for putting files until you or i know any better!

These are the programs/drivers required to get going:

gbDVB needs downloading and then extracting into /home/username/datv/ folder, it will create a folder called 'bin' in there.

Download Here Make sure you use V3.3 for 64bit OS or V3.1 for 32bit OS. The newer versions are more complicated so try and avoid them.

The gbDVB version 3.1 for 32bit has a slightly different command line requirement and will need some slight modification of the 1000 and 2000 files for it to work correctly. Email me if you require help here... i will try and generate some new script files and post them here soon for this version.....

ffmpeg needs to be installed using the Synaptic Package Manager in the system menu, this is like Add/Remove programs for windows but gives you a long list of programs available to downlaod and install. Search for ffmpeg and mark it for installation, click apply and then it will be installed.

script/ts files can be downloaded directly from me here These need to be extracted directly into the /home/username/datv/ folder, ther will be some .ts files and a couple of script (batch) files.

If you have two capture cards you can try these configuration files to generate a stream with two channels, i advise you to get the single card system working first!

The next two parts require you to compile source code... it's not as diffiicult and you may expect as long as your read the instructions properly. I will add some more hints after there have been some other people test it out.

To compile the code you need to have a terminal open and be in the folder where you have extracted the files, this is where you will type the commands that the help files talk about, only do this after you have installed the dependancies.

FT245 Drivers...Not required.... Update 22-Feb-2010

I have found just recently that you do not need the specific ft245 drivers for it to work, the pre-installed FTDI serial driver will work fine but as a different device name. This applies to both 32bit and 64bit systems.

Instead of being /dev/ft2450 it will be /dev/ttyUSB0. You will need to change this near the top of the 1000 and 2000 script files.

If you have already installed the specific ft245 driver then please ignore this update!

FT245 drivers for linux will need to be installed (this driver does NOT work with Ubuntu 9.10)

How to install the drivers This is very useful and explains everything really.

FT245 Linux driver download link

Please confirm this is working before you continue to OpenCaster installation as it's usually the main problem.

I found i only had to install these dependancies (and not all of the ones described in the instructions) on Ubuntu 8.04 (64bit) for the drivers to compile properly. Use the 'Synaptic Package Manager' to install the packages below.

linux-headers-2.6.24-24-generic (these are the kernel header .h files and will depend on your version of Linux)

linux-libc-dev

Note: type uname -r in the terminal to find your version number.

I named the device as ft245 instead of sdriq so it's more recognisable and ties in with my script files.

When you have your usb connected you should see a file called ft2450 in the /dev folder, it will dissapear when you disconnect the usb.

OpenCaster2.2 needs to be downloaded from here you will need to register on the website to access the reserved area where the downloads are kept. Download the 'complete package' for OpenCaster2.2. Extract the gzip file into /home/username/datv/ folder, it will create a folder called 'OpenCaster2.2' in there.

There is more to do with OpenCaster before it can be used... It is downloaded in source code so it needs to be compiled into executable files before it can be used.

There is a file called INSTALL in the opencaster directory that explains how to install it, it talks about dependancies, these are other source code packages that it needs to be able to compile into executable files, these can be found in the 'Synaptic Package Manager'

I found i only needed to install these dependancies for it to compile.

python2.5-dev

libc6-dev

zlib1g-dev

Once these are installed you are pretty much there!

You should be able to run the 2000 script file (in a terminal by typing ./2000) in the datv folder and you should get activity on the hardware. If you get some errors you can try and figure out what the problem is or email me the errors and i'l try figure it out!

How it all works....

PVR-150 usage:

We need to understand how to control the PVR-150 card with Video For Linux 2 (v4l2). This is the driver for most video devices in linux and is able to set the parameters on the capture card, we need to setup the MPEG-2 settings before we can get any usable signal from the encoder.
You should not need to install any other drivers from the pvr-150 card, the OS will already have them installed. (ok using ubuntu 8.04 and above i believe)
The main settings we will use are stream type, aspect ratio, video bitrate, audio bitrate, audio mode, input channel and resolution. These can be set with the command v4l2-ctl and the parameter setting, i have created a script to run these before we start using the data from the encoder.
The data that is output from the encoder is in Program Stream format and to get this into seperate elementry streams i'm now using ffmpeg which seems to have less delay in processing than vlc.
The reason we do not use ffmpeg's built in TS muxer is becaue we cannot change any of the TS format like channel name and provider name without re-compiling the code and apparently it does not work correctly anyway!

OpenCaster tool usage:

The Audio and Video elementry streams are converted into separate program streams using esaudio2pes and esvideo2pes from OpenCaster tools.
Then both of the program stream files are converted into transport stream files using pesaudio2ts and pesvideo2ts, these will be muxed together using tscbrmuxer later.
Using both of these programs will give us far more reliable stream to send out through the transmitter than going directly from VLC or ffmpeg's TS muxers.

Service Information:

Next we need to generate the related files to add to the blank transport streams to make it into a DVB transport stream, this is done with part of the OpenCaster suite using a python script. In case you have never heard of it python is another type of programming code.
The script generates separate .ts files with each of the parts of the DVB stream data. The files are PAT.ts, PMT.ts, SDT.ts, NIT.ts, EIT.ts, TDT.ts
You can change this script to add your own details like Channel name, Provider name and set the current programme information for a channel. This information can be changed while the stream is running as tscbrmuxer reads the files at regular intervals so you could update the program information in real time!

Changing these parameters is not too complicated but take care when making any changes. The file that needs to be modofied is in the datv folder with the other files and called si_gen.
Inside the file you change the Program name(channel name), Provider name, and Program information.
Once you have made these changes you need to open a terminal then cd to /home/username/datv then type ./si_gen which runs the script and creates the new .ts files
You may get some warnings after this but should NOT get any errors.
In the datv folder you should now have updated DVB .ts files (check the file date/time created info to be sure) and you will now have updated the stream information with your callsign etc…

Final Muxing:

Now we have our Audio+Video TS and we have generated the DVB information TS data files we need to use another OpenCaster tool called tscbrmuxer, this program takes all of the TS files and muxes them together into one single DVB transport stream ready for sending to gbDVB software.

gbDVB sends the final stream via a symbolic link to the /dev/ft2450 device and out to the transmitter via the usb port.

I would still like to note that you need quite a fast PC to be able to do all this procesing in real-time, it uses 45% of CPU on my AMD64 4200 Dual core system while running 2Ms/s rate and about 35% CPU usage running 1Ms/s rate.

Some tips for using Ubuntu..

Video4Linux2 - Some useful commands you can type in the terminal..
v4l2-ctl --list-devices - Lists what video devices are connected to the pc
v4l2-ctl --list-inputs - Lists the inputs available like composite,tuner etc.

The terminal
The terminal is just like the windows dos console where you can type commands in.
If you open the terminal the default path is your home folder (/home/username).
'/' is equivalent to 'C:\' as the root directory of the drive.
'dir' lists the files in the current directory, you can also use 'ls'.
'cd ..' moves you up one directory from the current path.
'cd dirname' would move you into the folder 'dirname' in the current path.
'rm filename' will delete the file.

Access Privileges
Typing 'sudo command' in the terminal will run the program 'command' with root (administrator) privileges, you will need this when using the make install command like this: 'sudo make install' You will be prompted to type your password in to allow this to run.
As default the file browser (nautilus) will only have limited access to the main file system, if you need more access like when placing the ft245 driver file and configuring it then you can use this command: 'sudo nautilus' and it will bring up the file browser with root privileges.
You can change to root user in the terminal by typing 'sudo -s' but it is far better to use sudo then the command name instead.

General
The System Monitor shows what processes are running just like windows task manager.
Add/Remove programs is done with the Synaptic Package Manager in system-administrator menu, find the package you need, check the box next to it to mark for installation. Once you have selected one or more packages you click the Apply button and it will install the ones you have selected.
The Synaptic Package Manager needs internet access so it can download the packages and install them, if you don't have this there will be a limited list to install and you may not find what you are looking for. Clicking the reload button while connected to the internet will load an up to date list of packages that are available.
The home folder is just like Documents and Settings in windows.
If you plug a memory stick or similar into usb there will be a icon created on the desktop that links to it, double click to open.
Holding Alt and the function keys brings up different screens where you can see startup information or login to the terminal full screen and keep the desktop separate.
Alt-F7 brings you back to the desktop.
In the terminal you can type 'man programname' to bring the help/manual file up for the specified program, press q to exit the help. There is help information for most of the standard linux commands like sudo, rm, cd etc…

That's all for now!! please let me know what problems you have and i will update the information.

---------------------------------------------------------------------------------------------------------

First Live Transmissions in Windows 4th May 2009

The Linux solution above is now the way to go, this windows version is now for testing only and does not work that well.

(~2 seconds delay from real time..)

There are quite a few software/stream bit rate issues that cause some interruption of the video stream but it's a step forward!

A very welcome email from Stefano, IW3IJQ came into my inbox last week and he pointed me to some software that was able to convert an MPEG2 Transport Stream into a compatable file for the 'Updated' Poor Man's DATV Transmitter in real time :-)
The program is called gbdvb and is project by Giuseppe Baruffa, the program is capable of taking an MPEG-TS stream as input and outputs the IQ data which is compatable with the NEW serialiser.
I have spent a lot of time trying to make the USB Serialiser work with the output of the gbDVB program, most of the time taken due to my poor knowledge of C programming but I have finally got something that works!
The gbdvb program outputs the interleave data to a temporary file and i read this data into my USBtx program to send it to the serialiser via usb port.
The gbevb program needs some slight modification to get around using this temporary file - the author has told me it maybe available in a future release and will probably reduce the overall delay to under 2 seconds. There is a workaround for this problem in linux but i have not tested it yet."

The gbdvb suite is available here: gbDVB Windows direct download link here for version 3.3: gbDVB v3.3 Version 3.4 does not have any extra features, and has the annoying registration problem so i suggest you use version 3.3
VLC (VideoLAN) is available here: VideoLAN
The USB interface program I have made still has quite a few bugs but makes basic live video possible! the USBtx.exe is available here: USBtx.zip - Use at your own risk!! Updated 21st May

Drivers for the FTDI UM245 USB Interface are here

I have created some example batch files to start VLC encoding and the gbDVB to start producing the output file for the transmitter. This is here: batch files.zip Updated 21st May

You will need to modify the enclive.bat file and replace the parameters for directshow input for VLC (your camera+audio input) all other parameters will be fine as standard.

It currently only works at 1Ms/s symbol rate, new software that generates a proper DBV Transport Stream is needed to make any higher bitrate possible.

To make live encoding possible you need to have a video capture device available on your PC. I'm using a Hauppauge WinTV capture card but most USB webcam will work but with longer delay from real time.

You will need quite a fast pc to do Live DATV!

To check if your pc is fast enough there is a speed check program in the gbDVB package described above. You need to extract gbDVB files from the zip into a folder. Next go to that folder and run the speed_dvbse.bat file. This will show the encoder speed with various different encoder settings, the only thing you need to look at is the first check that comes up on the screen it will show something like this:

This is run on my pc, a dual core 2.2GHz Intel processor... it is capable of encoding upto 2000kb/s which is enough to do live DATV at 1Ms/s, maybe 2Ms/s but it's not 100% working yet. I would guess anything over 1000kb/s should be ebough to make 1Ms/s live DATV. The following tests were under Windows XP, better results found when running Linux.

The dvbsenco.exe file (gbDVB), USBtx.exe and enclive.bat files all need to be placed in VideoLAN's main working directory
probably: "C:\Program Files\VideoLan\VLC\"

You will need to modify the enclive.bat file and replace the parameters for directshow input to VLC (your camera+audio input) all other parameters will be fine as standard.

To start it all running:
Run enclive.bat - VLC should load up and start encoding the live video input which should appear in the vlc display, gbDVB should also load in a dos window and show the parameters with it's encode process status at the bottom left increasing in value. You will also see a file being created in teh VLC directory called 'test_ci.bin' this is the 'pipe' file.

Once the above is up and running start USBtx.bat - make sure the USB serialiser is the only FTDI type device connected to usb ports as it needs to be device 0! USBtx will load another dos window showing the transfer to the serialiser and the approx buffer delay due to the 'pipe' file.

The limiting factor at present is the output bitrate from VLC, it varies quite a lot depending on the video content and with the serialiser being fixed bitrate the data to send either backs up or underruns in my USBtx program…. This is the biggest problem left to solve i think.

** The serialiser has the FEC viterbi coder built in and will only do 1/2 FEC rate but at 2Ms/s and 1/2 the data rate is good at ~1800Kb/s so will provide quite good video once we have some new software.

Lets hope there is a lot more Digital ATV activity now Live video is possible at the price of a little construction!

Here is a short video clip of it in action.. Live DATV demo

Some test files for sending directly to the serialiser with USBtx.exe - Test Files 13MB zip (1000/2000Ms/s)

Useful utilities:

Mpeg Bitrate Viewer - Graphs bitrate of an MPEG file

TsMuxeR - Will actually generate a constant bitrate MPEG2 Transport Stream from another file - It's picky with filetypes but will convert program stream files (.mpv,.mpa) nicely. Make sure the combined bitrate of the Input files is a bit less than the output bitrate (921Kbps for 1000 symbol rate)

MPEG2 Packet Analyser - Let's you view the contents of each TS Packet

MainConcept Reference - Video enocder for various formats, has DVB format for output so this can generate good CBR streams. Free demo has a logo in the corner...but you could live with that ;-)