[Xmltv-devel] RFC _fr_kazer: testing the Static File Grabber concept with kazer.org

[Karl Dietz]

Hi Kazer and fellow developers,

I have written a grabber that downloads the guide from kazer.org and
wraps it into a real grabber with apiconfig, etc.
For the general idea behind the concept see:
http://wiki.xmltv.org/index.php/User:Dekarl/Static_File_Grabber_Template

The purpose of this grabber is to make configuration easier than
explained at http://mythtv-fr.org/wiki/xmltv-kazer, which at the least
need the user to find the guide in the first place.

My intention with the grabber is to get a nice example of how easy it
is to write a full featured grabber for the existing xmltv listing
sites that don't have one yet.
See http://wiki.xmltv.org/index.php/User:Dekarl#Potential_Data_Sources

I'm using the supplementary files code to download the guide.zip and
reuse the caching behaviour while keeping the grabber code simple.
Do you think this is a sensible approach to take?


Unless we have a release coming up the next days I'd like to commit the
grabber and start refactoring the common parts that will be the same in
the next grabber into a shared module.

Regards,
Karl

[Ben Bucksch]

Post by Karl Dietz
Hi Kazer and fellow developers,
I have written a grabber that downloads the guide from kazer.org and
wraps it into a real grabber with apiconfig, etc.
http://wiki.xmltv.org/index.php/User:Dekarl/Static_File_Grabber_Template

I have problems to understand what you mean with "Static File Grabber"
and what exact format the site offers.

Post by Karl Dietz
If you already have your guide file on the internet and want to
support xmltv consumers you can either convert it upstream on your
webserver

This seems to suggest to implement a grabber on a webserver, then offer
xmltv files from your own server, based on third party data. That is
typically a copyright violation. You need to make that paragraph
clearer, what you mean.

Post by Karl Dietz
I'm using the supplementary files code to download the guide.zip

I believe that Supplementary Files MUST NOT contain executable code
(including interpreted code), because that is a security threat for all
users. If the server is hacked, all users are hacked. The server is not
properly secured for that, and not intended for that. Code updates
should be delivered via the distro packages.

Ben

[dekarl]

Hi Ben,

Post by Ben Bucksch

Post by Karl Dietz
I have written a grabber that downloads the guide from kazer.org and
wraps it into a real grabber with apiconfig, etc.
http://wiki.xmltv.org/index.php/User:Dekarl/Static_File_Grabber_Template

I have problems to understand what you mean with "Static File Grabber"
and what exact format the site offers.

I'm trying to make it easy to wrap a full featured grabber around existing guide
services that, more or less, make one regularly updated static xmltv file available on
a webserver.

Kazer is a xmltv guide service from the french community that has been around for
years, but they have not contributed a grabber to the XmltvProject yet.
After registration on their site you can download their tvguide in one big file in xmltv
format inside of a zip container.

Post by Ben Bucksch

Post by Karl Dietz
If you already have your guide file on the internet and want to
support xmltv consumers you can either convert it upstream on your
webserver

This seems to suggest to implement a grabber on a webserver, then offer
xmltv files from your own server, based on third party data. That is
typically a copyright violation. You need to make that paragraph
clearer, what you mean.

I am talking about services like Radio Times, SWEDB, to name examples which
already have grabbers in the source tree. Swedb does the conversion to xmltv
on their servers and Radio Times have a custom format that gets converted by
the grabber.
True, there are dumps of EPG data on strange anonymous file hosting services,
too. But that is not what I am targetting.

Post by Ben Bucksch

Post by Karl Dietz
I'm using the supplementary files code to download the guide.zip

I believe that Supplementary Files MUST NOT contain executable code
(including interpreted code), because that is a security threat for all
users. If the server is hacked, all users are hacked. The server is not
properly secured for that, and not intended for that. Code updates
should be delivered via the distro packages.

I agree. "The code" is Supplement.pm itself, not downloaded via Supplement.pm.
I want to reuse the codebase of supplemental files to avoid writing duplicate
code for caching a file from a server yet another time. So "the server" is the
kazer.org server in this case.

That might be a good target for refactoring to have two interfaces (Supplemental
File Download and Guide Data Download) to one shared functionality. But that is
a future step.


I hope this clears it up a bit

Regards,
Karl

___________________________________________________________
Schon gehört? WEB.DE hat einen genialen Phishing-Filter in die
Toolbar eingebaut! http://produkte.web.de/go/toolbar