[Xmltv-devel] Updating the IceTV grabber for possible inclusion in the XMLTV project

[Daniel Hall]

Hi Everyone,

Just wanted to introduce myself. I am the CTO of the Australian EPG
provider IceTV.

We have been upgrading our XMLTV feed and even have a basic XMLTV
subscription now which doesn't include our remote scheduling and advanced
recording options for a reduced price.

We want to see if we can get a new grabber added to the XMLTV distribution
to make it easier for people to use IceTV, with that in mind I do have some
questions that I hope you guys can answer.

With the grabber are the 'days' and 'offset' options mandatory? We
generally provide the entire feed at once but then can also just provide
incremental updates of changed guide data (using the 'fast' parameter).

*Daniel Hall*
*CTO *
* <http://www.icetv.com.au>*
*visit me on: [image:
Loading Image...]
<http://au.linkedin.com/pub/daniel-hall/1/440/7b/>*119 Willoughby
Road, Crows Nest, NSW, 2065.
PO Box 240, Wentworth Falls, NSW, 2782, Australia.

[Nick Morrott]

Post by Daniel Hall
Hi Everyone,
Just wanted to introduce myself. I am the CTO of the Australian EPG provider IceTV.
We have been upgrading our XMLTV feed and even have a basic XMLTV subscription now which doesn't include our remote scheduling and advanced recording options for a reduced price.

Hi Daniel,

Welcome to the XMLTV project, and thank you and IceTV for providing an
XMLTV listings feed to users.

Post by Daniel Hall
We want to see if we can get a new grabber added to the XMLTV distribution to make it easier for people to use IceTV, with that in mind I do have some questions that I hope you guys can answer.

Great :)

Post by Daniel Hall
With the grabber are the 'days' and 'offset' options mandatory? We generally provide the entire feed at once but then can also just provide incremental updates of changed guide data (using the 'fast' parameter).

Those options are mandatory iff the grabber claims support of the
"baseline" capability [1].

The baseline capability is ideally that - the minimum options that a
grabber provides to an end-user to control output of their configured
channels and data. I'd hope that all grabbers support at least this
capability.

If you haven't already found them, I'd recommend looking through both
the "XMLTV Capabilities" [1] and "How to Write a Grabber" [2] articles
on the XMLTV wiki to get a grounding in which capabilities are
supported and how to advertise them in a grabber to
end-users/applications.

[1] http://wiki.xmltv.org/index.php/XmltvCapabilities
[2] http://wiki.xmltv.org/index.php/HowtoWriteAGrabber

Note that there is also the "preferredmethod" capability that can hint
to end-users and consuming applications the preferred (to the
supplying service) way to retrieve data i.e. if it is faster to
download all available data and filter locally in the grabber, or
specifically ask for certain days/offsets at runtime.

Please feel free to ask for more details if this doesn't answer your
question, and any more questions you might have.

Cheers,
Nick

[Daniel Hall]

Hi Nick,

Finally getting a chance to work on this one again, and did have a couple
more questions.

Firstly, I have updated the grabber and our web API to support the days and
offset usage.

In our feed we have included a custom DTD which is an extension of the
XMLTV DTD adding a couple of elements and defining a couple of new
episode-num schemes, however when attempting to run the tv_validate_grabber
script it comes back with validity errors on these new elements. Does the
grabber script only use the official XMLTV DTD? and if so is there a way to
extend that in any official way?

My other concern is that IceTV is a paid service, so an email address and
password is required to setup the grabber (we have some explanation of this
in the configuration steps), but for ongoing validation without an account
it would not be able to be configured. Has this come up in the past with
other grabbers, and is there a possible way around this?

*Daniel Hall*
*CTO *

* <http://www.icetv.com.au>*
*direct: [phone]mobile: [phone]visit me on: [image:
http://www.feng-gui.com/images/linkedin_logo_16x16.jpg]
<http://au.linkedin.com/pub/daniel-hall/1/440/7b/>*119 Willoughby
Road, Crows Nest, NSW, 2065.
PO Box 240, Wentworth Falls, NSW, 2782, Australia.

Post by Daniel Hall

Post by Daniel Hall
Hi Everyone,
Just wanted to introduce myself. I am the CTO of the Australian EPG

provider IceTV.

Post by Daniel Hall
We have been upgrading our XMLTV feed and even have a basic XMLTV

subscription now which doesn't include our remote scheduling and advanced
recording options for a reduced price.
Hi Daniel,
Welcome to the XMLTV project, and thank you and IceTV for providing an
XMLTV listings feed to users.

Post by Daniel Hall
We want to see if we can get a new grabber added to the XMLTV

distribution to make it easier for people to use IceTV, with that in mind I
do have some questions that I hope you guys can answer.
Great :)

Post by Daniel Hall
With the grabber are the 'days' and 'offset' options mandatory? We

generally provide the entire feed at once but then can also just provide
incremental updates of changed guide data (using the 'fast' parameter).
Those options are mandatory iff the grabber claims support of the
"baseline" capability [1].
The baseline capability is ideally that - the minimum options that a
grabber provides to an end-user to control output of their configured
channels and data. I'd hope that all grabbers support at least this
capability.
If you haven't already found them, I'd recommend looking through both
the "XMLTV Capabilities" [1] and "How to Write a Grabber" [2] articles
on the XMLTV wiki to get a grounding in which capabilities are
supported and how to advertise them in a grabber to
end-users/applications.
[1] http://wiki.xmltv.org/index.php/XmltvCapabilities
[2] http://wiki.xmltv.org/index.php/HowtoWriteAGrabber
Note that there is also the "preferredmethod" capability that can hint
to end-users and consuming applications the preferred (to the
supplying service) way to retrieve data i.e. if it is faster to
download all available data and filter locally in the grabber, or
specifically ask for certain days/offsets at runtime.
Please feel free to ask for more details if this doesn't answer your
question, and any more questions you might have.
Cheers,
Nick
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
xmltv-devel mailing list
https://lists.sourceforge.net/lists/listinfo/xmltv-devel

[Robert Eden]

Post by Daniel Hall
In our feed we have included a custom DTD which is an extension of the
XMLTV DTD adding a couple of elements and defining a couple of new
episode-num schemes, however when attempting to run the
tv_validate_grabber script it comes back with validity errors on these
new elements. Does the grabber script only use the official XMLTV DTD?
and if so is there a way to extend that in any official way?

Can you be a bit more specific on what your DTD changes? I wouldn't
think a episode_num name would be validated. Changing the *format* of a
value would, because it could break apps.

Post by Daniel Hall
My other concern is that IceTV is a paid service, so an email address
and password is required to setup the grabber (we have some
explanation of this in the configuration steps), but for ongoing
validation without an account it would not be able to be configured.
Has this come up in the past with other grabbers, and is there a
possible way around this?

The Schedules Direct grabbers ( tv_grab_na_dd, tv_grab_zz_sdjon,
tv_grab_zz_sdjson_sqlite) solve this by creating a creating an account
for the XMLTV validation app. The config file is not part of the
distribution and so the password is only known to the person who runs
the tester.. (which is Chris Butler, but seems to be down. I think Nick
was looking into taking it over)

Robert

[Nick Morrott]

Post by Robert Eden

Post by Daniel Hall
My other concern is that IceTV is a paid service, so an email address and
password is required to setup the grabber (we have some explanation of this
in the configuration steps), but for ongoing validation without an account
it would not be able to be configured. Has this come up in the past with
other grabbers, and is there a possible way around this?

The Schedules Direct grabbers ( tv_grab_na_dd, tv_grab_zz_sdjon,
tv_grab_zz_sdjson_sqlite) solve this by creating a creating an account for
the XMLTV validation app. The config file is not part of the distribution
and so the password is only known to the person who runs the tester.. (which
is Chris Butler, but seems to be down. I think Nick was looking into taking
it over)

Daniel/Robert,

I have taken over and reworked the regular testing of our suite of
XMLTV grabbers from Chris Butler, including the account-based
Schedules Direct grabbers, and run the userland testing locally in
Docker using Debian stable/testing/unstable containers. Account
usernames/passwords are not exposed, and allow for the testing of paid
grabbers if I am provided with a test account.

I haven't started putting recent userland results online as I've been
tinkering with the output, but I'll get this remedied.

Buildt-time testing is now carried out automatically using Travis CI
on my personal github mirror of the XMLTV CVS repository, whenever new
commits are synced. Build testing is currently checked against Debian
(stable/testing/untstable), Ubuntu (14.04 LTS, 16.04 LTS, latest),
Fedora (latest, latest-1) and CentOS (latest, latest-1) distributions.

Hope this helps,
Nick