Welcome to World of IPTV

With

+23k members
+11k threads
+106k posts

we are the most popular IPTV community on the web. 

IMPORTANT NOTE:
WE HAVE RECENTLY NOTICED THAT TOO MANY DOUBLE FAKE ACCOUNTS ARE CREATED IN THE PAST.
TO PREVENT THIS ISSUE THE DECISION WAS MADE THAT IN THE FUTURE A ANNUALLY FEE 20 EURO WILL BE RAISED FOR NEW MEMBERSHIPS.

Join now to the World of IPTV

Forum Rules

Before you start, check out the forum rules first

Account upgrade

Upgrade your account to get access to full features

Advertising

Would you like to place your advertisement with us ?

Resources Manager

Hundreds of IPTV scripts and apps are available for download

Info TSDuck, The MPEG Transport Stream Toolkit

doc

Banned
Banned
Ext. Member
Joined
Dec 27, 2019
Messages
143
Reaction score
689
Points
104
Location
@home
TSDuck is an extensible toolkit for MPEG/DVB transport streams. TSDuck is used in digital television systems for test, monitoring, integration, debug, lab, demo.

In practice, TSDuck is used for:

  • Transport stream acquisition or transmodulation, including DVB, ATSC, ASI and IP multicast.
  • Analyze transport streams, PSI/SI signalization, bitrates, timestamps.
  • Monitor and report conditions on the stream (video and audio properties, bitrates, crypto-periods, signalization).
  • On-the-fly transformation or injection of content and signalization.
  • Modify, remove, rename, extract services.
  • Work on live transport streams, DVB-S/C/T, ATSC, ASI, IP-TV, HTTP, HLS or offline transport stream files.
  • Receive from or send to specialized hardware such as:
    • Cheap DVB or ATSC tuners (USB, PCI).
    • Professional Dektec devices, ASI, modulators (USB, PCI).
    • HiDes modulators (USB).
  • Re-route transport streams to other applications.
  • Extract or inject Multi-Protocol Encapsulation (MPE) between TS and UDP/IP.
  • Analyze and inject SCTE 35 splice information.
  • Extract specific encapsulated data (Teletext, T2-MI).
  • Emulate a CAS head-end using DVB SimulCrypt interfaces to and from ECMG or EMMG.
  • And more...
TSDuck is developed in C++ in a modular architecture. It is easy to extend it through plugins. In fact, the primary requirement for TSDuck architecture was to be able to implement any new feature within a couple of hours, which happened to be the case most of the time.

To be easily extended, TSDuck needed to be simple. TSDuck is a collection of command line tools and plugins. There is no sophisticated GUI. Each utility or plugin performs only one elementary feature but they can be combined in any order.

In short, TSDuck shall be understood as a General-Purpose Toolbox for Digital TV Engineers and not as a specialized application for production operators.

And, by the way, TSDuck is Free and Open Source.

TSP, the transport stream processor
The most important TSDuck tool is tsp, the transport stream processor. It is a flexible transport stream processing framework. Running tsp is a combination of elementary processing operations using plugins.

In each invocation of tsp, there are:

  • One input plugin which receives a TS from various possible sources.
  • Any number of packet processing plugins which can perform any transformation on the packets in transit.
  • One output plugin which sends the TS to various possible destinations.
This is illustrated in the diagram below:
tsp.png


The currently existing input and output plugins can work on TS files, UDP/IP (multicast or unicast), DVB sources using cheap tuners, professional Dektec modulators and ASI devices, HiDes modulators.

Example: MPE injection and extraction
This example describes a test bed or demo infrastructure for MPE injection and MPE extraction.

The network infrastructure is illustrated in the diagram below.
mpedemo.png



In network 1, a media server multicasts a transport stream on address 224.250.250.1, port 9000. We want to encapsulate this UDP multicast stream in an existing transport stream using MPE. This existing TS is a live satellite feed which is received using a DVB tuner.

We also change the multicast destination address for the UDP stream to 230.2.3.4, port 7000, in the MPE-encapsulated datagrams. There is no particular reason for this, we just illustrate the feasibility.

The resulting transport stream with embedded MPE is then broadcast. Here, the broadcast network is a Dektec modulator, followed by another computer using a DVB tuner.

This computer is connected to a second network. Another instance of tsp extracts the datagrams from the MPE stream and multicasts them on its network using the modified destination address.

The first tsp command adds two services in the existing transport stream. One service carries the IP/MAC Notification Table (INT). The other one carries the MPE stream. We create a fully DVB-conformant signalization for these two services.

The second tsp command extracts the MPE stream from the received TS and forwards all decapsulated UDP multicast datagrams on the local network.

This example is fully described in the TSDuck User's Guide with all details and corresponding commands.

Merging and forking transport streams
Specific plugins such as merge and fork respectively combine and duplicate transport streams. They are designed to route transport streams from and to other applications. When the "other" application is another instance of tsp, we can create complex processing graphs.

This is illustrated in the diagram below.
mergefork.png


In addition to tsp, the command tsswitch can be used to switch inputs between a set of transport streams using the same input and output plugins as tsp. Switching between inputs can be performed automatically at the end of each input or through a remote control application. The remote control protocol is extremely simple, based on text messages in UDP datagrams.

This is illustrated in the diagram below.
tsswitch.png


The remote control can be performed using simple command lines. The bash shell provides an easy way to redirect output to an UDP message. The following sample commands send UDP messages on port 4444 to system 127.0.0.1 (the local host). This is the easiest way to use the tsswitch remote control.

echo >/dev/udp/127.0.0.1/4444 2
echo >/dev/udp/127.0.0.1/4444 next

Example: A CAS test bed
The following diagram illustrates a real-life example. To test a Conditional Access System and signalization, tsp is used to create a new transport stream from a live source. One real TS is received as input using a cheap USB or PCI DVB-T tuner. This TS and all its services are renamed, creating a "new TS" in the network. To test the CAS, tsp interacts with a real ECMG and a real EMMG using the standard DVB SimulCrypt protocols. One service, initially in the clear, is scrambled. tsp generates the control words, scrambles the content and inserts ECM's and EMM's. Finally, a set-top box receives all live transponders plus the new one we create.
castest.png


Everything is done in one single tsp process. The command is the following. For clarity, the details such as tuning parameters, access criteria, IP addresses and ports, are represented as environment variables.

Bash:
tsp -I dvb -u $UHF_INPUT                                      \
    -P tsrename -t 9 -a                                       \
    -P svrename direct8 -i 0x0901 -l 41 -n "Direct 8 Test"    \
    -P svrename bfmtv -i 0x0903 -l 42 -n "BFM TV Test"        \
    -P svrename 'i>tele' -i 0x0904 -l 43 -n "i>TELE Test"     \
    -P svrename virgin17 -i 0x0905 -l 44 -n "Virgin 17 Test"  \
    -P svrename gulli -i 0x0906 -l 45 -n "Gulli Test"         \
    -P svrename france4 -i 0x0907 -l 46 -n "France 4 Test"    \
    -P svrename 0x02FF -i 0x09FF                              \
    -P scrambler GulliTest -e $ECMG -s $SUPER_CAS_ID          \
         -p $PMT_CADESC_PRIVATE -a $AC                \
         -b $ECM_BITRATE --pid $ECM_PID               \
    -P cat -c -a $CAS_ID/$EMM_PID/$CAT_CADESC_PRIVATE         \
    -P datainject -r -s $MUX_SERVER_PORT                      \
          -b $EMM_MAX_BITRATE -p $EMM_PID             \
    -O dektec -u $UHF_OUTPUT --convolution 2/3 --guard 1/32

This command is complex but rational. The option -I on the first line specifies the input plugin and its arguments. Here, the input plugin is dvb which drives DVB tuners. Each option -P specifies a packet processor plugin and its options. The plugins tsrename and svrename respectively rename the transport stream and a service. The plugin scrambler is a DVB scrambler on one service. It generates control words, interacts with a standard ECMG and insert ECM's. It also updates the PMT of the service to declare the ECM PID. The plugin cat modifies or creates the CAT to declare a new EMM stream. The plugin datainject implements a DVB SimulCrypt EMM/PDG inserter. It is used as a server by a standard EMMG to insert EMM's, just as if it was a MUX. Finally, the option -O on the last line specifies the output plugin and its arguments. Here, the output plugin is dektec which drives a Dektec modulator, a PCIe board in the computer.

For more details, refer to the TSDuck User's Guide.

Of course, such a complex command should be maintained and edited in a shell script. Unless of course we enjoy retyping it every time.

Because of its modular structure, TSDuck is better used with a flexible shell. On Windows systems, we may use PowerShell, Cygwin or MSys (DOS command line windows should now be reserved to polymorphic deviants only).

The PSI/SI table compiler
TSDuck provides many ways to manipulate PSI/SI and other types of MPEG tables. Tables can be extracted, examined, injected or modified on the fly using specific plugins for tsp.

Tables can be manipulated in several formats:

  • Transport stream files: Sections and tables are extracted ("depacketized") from or inserted ("packetized") into TS packets in transport streams.
  • Section files: These binary files contain sections. Each section is stored in raw binary format. Sections are simply concatenated without any encapsulation.
  • XML files: TSDuck defines a specific XML syntax to represent most standard PS/SI tables and descriptors. XML being a text format, such tables can be edited or created by hand using any text editor.
TSDuck utilities manipulate tables in any of these formats and converts between them. Specifically, the command tstabcomp is a table compiler. It takes XML source files as input, compiles the tables they contain and output binary section files. Thus, we can create any arbitrary table from scratch and build the corresponding binary sections to inject them in a transport stream. The command tstabcomp is also a table decompiler which can recreate the XML representation of tables from their binary sections.

More generally, all TSDuck commands can manipulate tables in binary or XML format equally. Tables can be captured from a live transport stream and directly saved in XML format. We can then edit the XML files to modify the tables and reinject them in any other transport stream.

Sample XML source file
The expected XML input format is fully documented in the TSDuck User's Guide. Briefly, an input XML file uses <tsduck> as root node. The root node contains any number of tables. The following sample XML file contains the definition for simple (and incomplete) PAT and PMT:

XML:
<?xml version="1.0" encoding="UTF-8"?>
<tsduck>

  <PAT version="8" transport_stream_id="0x0012" network_PID="0x0010">
    <service service_id="0x0001" program_map_PID="0x1234"/>
    <service service_id="0x0002" program_map_PID="0x0678"/>
  </PAT>

  <PMT version="4" service_id="0x0456" PCR_PID="0x1234">
    <CA_descriptor CA_system_id="0x0777" CA_PID="0x0251"/>
    <component elementary_PID="0x0567" stream_type="0x12">
      <CA_descriptor CA_system_id="0x4444" CA_PID="0x0252"/>
      <ISO_639_language_descriptor>
        <language code="fre" audio_type="0x45"/>
        <language code="deu" audio_type="0x78"/>
      </ISO_639_language_descriptor>
    </component>
  </PMT>

</tsduck>

Let's look at the various steps in a real example. We need to reproduce the case of a broadcast stream (satellite, terrestrial, whatever) with a specially crafted NIT. The plugin nit in tsp provides a convenient way to modify a NIT on the fly. But let's assume that the modifications we need are so specific that this plugin cannot do what we want.

First, we capture the original NIT from the live stream and save it in XML format:

Bash:
tsp -I dvb --uhf 24 \
    -P tables --pid 16 --tid 0x40 --max 1 --xml nit.xml \
    -O drop

Now, we manually edit the XML file using any text editor and apply the specific modifications we need. Then, we can directly inject it in the stream:

Bash:
tsp -I dvb --uhf 24 \
    -P inject nit.xml --pid 16 --replace --stuffing \
    -O dektec --uhf 24 --convolution 2/3 --guard 1/32

Using TSDuck as a library
TSDuck can also be used as an MPEG/DVB C++ library for your own applications. Most of the TSDuck code is located into a large shareable library. The utilities and plugins are usually only small wrappers around the TSDuck library. You may use the TSDuck library to develop new plugins or create completely independent applications.

The following diagram illustrates the TSDuck software architecture.


All C++ classes in the TSDuck library are documented using Doxygen.

There is a binary installer for the TSDuck development environment too. So, there is no need to rebuild TSDuck first. See more details here.

Commands and plugins
Here is a summary of all command line utilities and tsp plugins which are installed with TSDuck version 3.20-1556.

TSDuck commands
tsanalyzeAnalyze the structure of a transport stream
tsbitrateEvaluate the bitrate of a transport stream
tscmpCompare two transport stream files
tsdateExtract the date and time (TDT/TOT) from a transport stream
tsdektecControl Dektec devices
tsdumpDump and format MPEG transport stream packets
tsecmgMinimal generic DVB SimulCrypt-compliant ECMG
tsemmgMinimal generic DVB SimulCrypt-compliant EMMG
tsfixccFix continuity counters in a transport stream
tsftruncTruncate an MPEG transport stream file
tsgenecmGenerate one ECM using any DVB SimulCrypt compliant ECMG
tshidesList HiDes modulator devices
tslsdvbList DVB tuner devices
tspMPEG transport stream processor using a chain of plugins
tspacketizePacketize PSI/SI sections in a transport stream PID
tspcontrolSend control commands to a running tsp
tspsiExtract all standard PSI from an MPEG transport stream
tsresyncResynchronize a non-standard or corrupted MPEG transport stream
tsscanScan a DVB network
tssmartcardList or control smartcards
tsstuffAdd stuffing to a transport stream to reach a target bitrate
tsswitchTS input source switch using remote control
tstabcompPSI/SI tables compiler
tstabdumpDump PSI/SI tables, as saved by tstables
tstablesCollect PSI/SI tables from an MPEG transport stream
tsterinfoCompute or convert DVB-Terrestrial information
tsversionCheck version, download and upgrade TSDuck
TSP input plugins
craftBuild specifically crafted input packets
dektecReceive packets from a Dektec DVB-ASI or demodulator device
dvbDVB receiver device input
fileRead packets from one or more files
forkFork a process and receive TS packets from its standard output
hlsReceive HTTP Live Streaming (HLS) media
httpRead a transport stream from an HTTP server
ipReceive TS packets from UDP/IP, multicast or unicast
nullGenerate null packets
TSP output plugins
dektecSend packets to a Dektec DVB-ASI or modulator device
dropDrop output packets
fileWrite packets to a file
forkFork a process and send TS packets to its standard input
hidesSend packets to a HiDes modulator device
hlsGenerate HTTP Live Streaming (HLS) media
ipSend TS packets using UDP/IP, multicast or unicast
playPlay output TS on any supported media player in the system
TSP packet processor plugins
aesExperimental AES scrambling of TS packets
analyzeAnalyze the structure of a transport stream
batPerform various transformations on the BAT
bitrate_monitorMonitor bitrate for TS or a given PID
boostpidBoost the bitrate of a PID, stealing stuffing packets
catPerform various transformations on the CAT
clearExtract clear (non scrambled) sequences of a transport stream
continuityCheck or fix continuity counters on TS packets
countCount TS packets per PID
craftCraft specific low-level transformations on packets
cutoffSet labels on TS packets upon reception of UDP messages
datainjectDVB SimulCrypt data injector using EMMG/PDG <=> MUX protocol
decapDecapsulate TS packets from a PID produced by encap plugin
descramblerGeneric DVB descrambler
duplicateDuplicate PID's, reusing null packets
eitAnalyze EIT sections
encapEncapsulate packets from several PID's into one single PID
fileWrite packets to a file and pass them to next plugin
filterFilter TS packets according to various conditions
forkFork a process and send TS packets to its standard input
historyReport a history of major events on the transport stream
injectInject tables and sections in a TS
limitLimit the global bitrate by dropping packets
mergeMerge TS packets coming from the standard output of a command
mpeExtract MPE (Multi-Protocol Encapsulation) datagrams
mpeinjectInject an incoming UDP stream into MPE (Multi-Protocol Encapsulation)
muxInsert TS packets in a transport stream
nitPerform various transformations on the NIT
nitscanAnalyze the NIT and output a list of tuning information
patPerform various transformations on the PAT
patternReplace packet payload with a binary pattern on selected PID's
pcradjustAdjust PCR's according to a constant bitrate
pcrbitratePermanently recompute bitrate based on PCR analysis
pcrextractExtracts PCR, OPCR, PTS, DTS from TS packet for analysis
pcrverifyVerify PCR's from TS packets
pesAnalyze PES packets
pmtPerform various transformations on the PMT
psiExtract PSI Information
psimergeMerge PSI/SI from mixed streams
reduceReduce the TS bitrate by removing stuffing packets
regulateRegulate the TS packets flow based on PCR or bitrate
remapGeneric PID remapper
rmorphanRemove orphan (unreferenced) PID's
rmspliceRemove ads insertions from a program using SCTE 35 splice information
scramblerDVB scrambler
sdtPerform various transformations on the SDT
sectionsRemove or merge sections from various PID's
sifilterExtract PID's containing the specified PSI/SI
skipSkip leading TS packets of a stream
slicePass or drop packets based on packet numbers
spliceinjectInject SCTE 35 splice commands in a transport stream
stuffanalyzeAnalyze the level of stuffing in tables
svremoveRemove a service
svrenameRename a service, assign a new service name and/or new service id
t2miExtract T2-MI (DVB-T2 Modulator Interface) packets
tablesCollect PSI/SI Tables
teletextExtract Teletext subtitles in SRT format
timeSchedule packets pass or drop, based on time
timerefUpdate TDT and TOT with a new time reference
timeshiftDelay transmission by a fixed amount of packets
triggerTrigger actions on selected TS packets
tsrenameRename a transport stream
untilCopy packets until one of the specified conditions is met
zapZap on one service: Produce an SPTS containing only the specified service
Availability
TSDuck is open source and released under the BSD license. It is developed in C++ and designed to be portable. TSDuck is built and tested for Windows, Linux and macOS. Please see the list of useful links in the side menu.

Pre-built binary packages are available for Windows, Fedora, Ubuntu, CentOS/RedHat (Intel) and Raspbian (Raspberry Pi). See the Download section in the side menu.

On macOS, TSDuck is installed using the Hombrew packager. See the guidelines to install TSDuck using Homebrew.

Rebuilding TSDuck, with or without its binary installers and packages, is documented here.

Support for specialized hardware, DVB tuners, Dektec devices, HiDes modulators, is available on Windows and Linux only.

Please report problems or ask questions using the issue tracker.

References
TSDuck is referenced in the list of Open Source Software for Broadcasters by the EBU (European Broadcasting Union).

TSDuck was presented at the EBU Open Source Meetup during the IBC 2018.

An technical note from Cisco named "Configure DVB-C Lab Environment with cBR-8, TSDuck, and VLC" describes how to configure a Digital Video Broadcasting - Cable (DVB-C) lab scenario with the TSDuck toolkit, VLC, and cBR-8 (a Cisco equipment).

License
TSDuck is released under the terms of the license which is commonly referred to as 2-Clause BSD License or Simplified BSD License or FreeBSD License.

Copyright (c) 2005-2019, Thierry Lelégard
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
  1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
This software is provided by the copyright holders and contributors "as is" and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. in no event shall the copyright owner or contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage.
 
Channels MatchTime Unblock CDN Offshore Server Contact
100 cnx / 90€ 5Gbps / 180€ 48CPU-256GRAM 10Gbps 569€ Skype live:giefsl
500 cnx / 350€ 10Gbps / 350€ 48CPU-128GRAM 5Gbps / 349€ TG @changglobize
1000 cnx / 500€ 20Gbps / 700€ 40CPU-128GRAM 20Gbps / €980 http://coronaserver.com
shape1
shape2
shape3
shape4
shape5
shape6
Top
AdBlock Detected

We know, ad-blocking software do a great job at blocking ads. But our site is sponsored by advertising. 

For the best possible site experience please take a moment to disable your AdBlocker.
You can create a Account with us or if you already have account, you can prefer an Account Upgrade.

I've Disabled AdBlock