[Xmltv-devel] A new tv_grab_API

[Hika van den Hoven]

Hoi All,

I have mentioned before that since the start of this year I have been
working on turning the dutch grabber tv_grab_nl.py into a generic API
for creating a grabber. I have fully separated all grabber specific
data from the code, making it for a maintainer not necessary to touch
the code. Creating a grabber only takes a week and any changes on the
source(s) can be updated by the maintainer without the user needing to
upgrade. Only in case of lineup changes the user will have to re-run
configure in which case the maintainer can notify the users through
their log-file. Several test and debug scripts and modi are available
to the maintainer. Also all texts are placed outside the code and are
at present available in English and Dutch, but additional languages
can be easily added.
At present I am nearing the end of beta testing and am expecting the
end of beta somewhere next month, when I will release version 1.1. I
mostly have to further work on the documentation. Developer
documentation is 75% ready and user documentation mostly only still
exists in Dutch.
The code as of the current release 1.0.4, except of cause for fixes,
is frozen. I have many thoughts on extending functionality, but those
for now have to wait on later releases.

If you are interested see: https://github.com/tvgrabbers/tvgrabpyAPI
and the wiki there.

For the data-files I have developed a JSON based "data_def" language
that is contained in a separate module "DataTreeGrab"
https://github.com/tvgrabbers/DataTree
I must confess that I fell in love with it when in February this
"data_def" language started to take form.

The download URLs:
https://github.com/tvgrabbers/DataTree/releases/tag/beta-1.2.5-p20161015
https://github.com/tvgrabbers/tvgrabpyAPI/releases/tag/beta-1.0.4-p20161015
--
Tot Mails,
xmltv-***@lists.sourceforge.net mailto:***@gmail.com

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens

[Robert Eden]

Post by Hika van den Hoven
I have mentioned before that since the start of this year I have been
working on turning the dutch grabber tv_grab_nl.py into a generic API
for creating a grabber. I have fully separated all grabber specific
data from the code, making it for a maintainer not necessary to touch
the code.

I'm a little confused on what you've created. Is this a python
equivalent to xmltv.pm?

I looked at some of your documentation and you talk about creating a
grabber without knowing
any python. I'm not sure how that works... are you talking about
grabbing data from any
source already providing XMLTV formatted data or is there some other
file format you're
grabbing and converting.

Be sure and add your project to the Wiki at XMLTV.ORG

Robert

[Hika van den Hoven]

Hoi Robert,

Post by Robert Eden

Post by Hika van den Hoven
I have mentioned before that since the start of this year I have been
working on turning the dutch grabber tv_grab_nl.py into a generic API
for creating a grabber. I have fully separated all grabber specific
data from the code, making it for a maintainer not necessary to touch
the code.

I'm a little confused on what you've created. Is this a python
equivalent to xmltv.pm?

I do not know xmltv.pm, so I can not compare.
It's a data driven API. You create the JSON data files telling it
where and what to grab and it returns a nice xmltv file. DataTreeGrab
imports any html or json page into a node tree and you tell the
program how to crawl through that tree and at which points to extract
data returning it in a uniform format to the main API. The main API
merges the data from all the sources (11 at present for the Dutch
grabber) and returns the xmltv file. This whole proces is
manipulatable in a lot of ways by the maintainer and by the user.
By the first through the JSON data files and by the user through many
settable options.

Post by Robert Eden
I looked at some of your documentation and you talk about creating a
grabber without knowing
any python. I'm not sure how that works... are you talking about
grabbing data from any
source already providing XMLTV formatted data or is there some other
file format you're
grabbing and converting.
Be sure and add your project to the Wiki at XMLTV.ORG

I'm planning to

Post by Robert Eden
Robert
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
xmltv-devel mailing list
https://lists.sourceforge.net/lists/listinfo/xmltv-devel

Tot mails,
Hika Alina Maria van den Hoven mailto:***@gmail.com

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens
--

[Robert Eden]

I'm a little confused on what you've created. Is this a python

Post by Robert Eden
equivalent to xmltv.pm?

I do not know xmltv.pm, so I can not compare.
It's a data driven API. You create the JSON data files telling it
where and what to grab and it returns a nice xmltv file. DataTreeGrab
imports any html or json page into a node tree and you tell the
program how to crawl through that tree and at which points to extract
data returning it in a uniform format to the main API. The main API
merges the data from all the sources (11 at present for the Dutch
grabber) and returns the xmltv file. This whole proces is
manipulatable in a lot of ways by the maintainer and by the user.
By the first through the JSON data files and by the user through many
settable options.

xmltv.pm is a perl library with tools to make it easier to generate a
valid XMLTV formatted file. Based on your description, you have
something different.

So, to summarize your utility, it's a Python script that generates XMLTV
by parsing a web based JSON or XML (including HTML) source into a tree
and running that tree against a JSON based definition language to
mapping tree leaves to a XMLTV record.

Right?

Robert

[Hika van den Hoven]

Hoi Robert,

Post by Robert Eden

I'm a little confused on what you've created. Is this a python

Post by Robert Eden
equivalent to xmltv.pm?

I do not know xmltv.pm, so I can not compare.
It's a data driven API. You create the JSON data files telling it
where and what to grab and it returns a nice xmltv file. DataTreeGrab
imports any html or json page into a node tree and you tell the
program how to crawl through that tree and at which points to extract
data returning it in a uniform format to the main API. The main API
merges the data from all the sources (11 at present for the Dutch
grabber) and returns the xmltv file. This whole proces is
manipulatable in a lot of ways by the maintainer and by the user.
By the first through the JSON data files and by the user through many
settable options.

xmltv.pm is a perl library with tools to make it easier to generate a
valid XMLTV formatted file. Based on your description, you have
something different.
So, to summarize your utility, it's a Python script that generates XMLTV
by parsing a web based JSON or XML (including HTML) source into a tree
and running that tree against a JSON based definition language to
mapping tree leaves to a XMLTV record.
Right?
Robert

Basically. Only it's HTML including XML and it is a three, four step
process with one of those steps being the merging of multiple sources
and selective ttvdb lookup. The extraction is at first to a dict and
only the last step is the XMLTV creation, making it also possible to
add possible different output formats. It would also be relative easy
to add other extraction formats then JSON or HTML.

Tot mails,
Hika Alina Maria van den Hoven mailto:***@gmail.com

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens
--

[Hika van den Hoven]

Hoi Robert,

Post by Robert Eden
Be sure and add your project to the Wiki at XMLTV.ORG
Robert

Done that. See:
http://wiki.xmltv.org/index.php/XmltvRelatedProjects#tv_listing_grabbers_not_part_of_XMLTV_SF_project

Let me know if anything is unclear and feel free to ask or request
additions.

Tot mails,
Hika Alina Maria van den Hoven mailto:***@gmail.com

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens
--