SourceForge.net
2011-02-05 00:08:59 UTC
Feature Requests item #3135167, was opened at 2010-12-11 21:13
Message generated for change (Comment added) made by stefanb2
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=424138&aid=3135167&group_id=39046
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: tv_grab_fi
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Pekka Korpinen (pkorpine)
Assigned to: Stefan Becker (stefanb2)
Summary: tv_grab_fi: fetch information from tvnyt.fi
Initial Comment:
While visiting tvnyt.fi I noticed that its javascript uses JSON formatted data for the TV program information. Thanks to JSON, the parsing becomes a trivial task. I think it would be useful to take a look if it's worth to change the tv_grab_fi's source from telkku.com (which has become slow and awful) to tvnyt.fi.
I actually created today a simple Python script (I don't Perl, sorry) for myself that creates xmltv compliant file from tvnyt.fi's data.
http://iki.fi/~pkorpine/proj/2010/xmltv/
----------------------------------------------------------------------
Message:
Seiska offers another online TV guide http://www.telvis.fi. I've added that
as the fourth data source to the rewrite.
@dekari: I sent an email to YLE/Welho about 4 weeks ago, asking for
permission to use their data. I haven't got any responses. Should we now
assume "we're not OK with it" or "we don't care"? I'll resent the email to
also include Telkku & Telvis.
----------------------------------------------------------------------
Comment By: Stefan Becker (stefanb2)
Date: 2010-12-25 15:02
Message:
Courtesy of AirBerlin and Ms. Holle I spent plenty of hours on the
Helsinki-Vantaa airport and had time to progress the implementation a lot.
Over the last few days I finalized the rewrite and it should now support
all the features of the original grabber. If you want to have a peek, here
is the URL of the read-only git repository:
git clone http://chemobejk.dyndns.org/git/tv_grab_fi_rewrite.git
It currently supports the sources telkku.com, tvnyt.fi and yle.fi (finnish
& swedish [I hope this information is the same as svenska.yle.fi that
tv_grab_fi_sv uses]). Execute merge.pl to generate the old-style
one-script-grabber from the modular sources.
Including tvnyt.fi sources in test.conf currently still leads to test
failures. I haven't figured out yet how to handle the overlapping
information between the days so that it works correctly for a multiple-day
fetch and a single-day-then-merge fetch.
Hyvää Joulua!
----------------------------------------------------------------------
Comment By: Per Lundberg (perlundberg)
Date: 2010-12-12 23:05
Message:
Here's a thought: if/when you are implementing support for multiple
sources, why not merge in some of the code I've written in the
tv_grab_fi_sv grabber, to make it possible to grab both Swedish and Finnish
listings at once?
Since only some of the channels are available in Swedish, you have to
resort to use the tv_grab_combiner if you want to combine Swedish listings
for the channels where possible, with Finnish listings for the other
channels. If the tv_grab_fi could do it all... setting up this kind of
setup would be simplified a lot.
Please, just ask (perlun at gmail) if you have any questions regarding
the fi_sv grabber; you should be able to merge in code from there pretty
easily, I'd hope. (It uses the XML DOM parser to parse the HTML data, since
I find that preferable than setting up a lot of regexps.)
----------------------------------------------------------------------
Comment By: Karl Dietz (dekarl)
Date: 2010-12-12 19:34
Message:
I'd send a mail telling them that you'd like to make a program (as part of
xmltv) to use their welho tv guide on your (custom built) HTPC asking for
permission to do so and share the program.
----------------------------------------------------------------------
Comment By: Stefan Becker (stefanb2)
Date: 2010-12-12 17:42
Message:
@dekari: any suggestions how?
----------------------------------------------------------------------
Comment By: Karl Dietz (dekarl)
Date: 2010-12-12 14:06
Message:
@all: please, remember to get permission to use for each additional data
source.
@pkorpine: thanks for making your script API compatible
----------------------------------------------------------------------
Comment By: Ville Ahonen (va1210)
Date: 2010-12-12 11:18
Message:
Yes, this looks like a perfect addition to Stefan's rewrite of tv_grab_fi!
(And yes, "vapaa aika" == "recreational")
----------------------------------------------------------------------
Comment By: Stefan Becker (stefanb2)
Date: 2010-12-12 10:26
Message:
I'm currently working on a tv_grab_fi rewrite from scratch. One of the new
features I'm looking at is supporting multiple sources. I'll add tvnyt.fi
(or is it actually welho who provides this service???) to the list of
sources.
Thanks for the Python code. They have hidden getChannelPrograms.aspx very
well in the JavaScript code:-)
One good thing: the data contains a category which could be translated to
XMLTV information:
0 - unknown
1 - dokumentit (documentary)
2 - draama (drama)
3 - lapset (children)
4 - uutiset (news)
5 - urheilu (sports)
6 - vapaa aika (recreational?)
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=424138&aid=3135167&group_id=39046
Message generated for change (Comment added) made by stefanb2
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=424138&aid=3135167&group_id=39046
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: tv_grab_fi
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Pekka Korpinen (pkorpine)
Assigned to: Stefan Becker (stefanb2)
Summary: tv_grab_fi: fetch information from tvnyt.fi
Initial Comment:
While visiting tvnyt.fi I noticed that its javascript uses JSON formatted data for the TV program information. Thanks to JSON, the parsing becomes a trivial task. I think it would be useful to take a look if it's worth to change the tv_grab_fi's source from telkku.com (which has become slow and awful) to tvnyt.fi.
I actually created today a simple Python script (I don't Perl, sorry) for myself that creates xmltv compliant file from tvnyt.fi's data.
http://iki.fi/~pkorpine/proj/2010/xmltv/
----------------------------------------------------------------------
Comment By: Stefan Becker (stefanb2)
Date: 2011-02-05 02:08Message:
Seiska offers another online TV guide http://www.telvis.fi. I've added that
as the fourth data source to the rewrite.
@dekari: I sent an email to YLE/Welho about 4 weeks ago, asking for
permission to use their data. I haven't got any responses. Should we now
assume "we're not OK with it" or "we don't care"? I'll resent the email to
also include Telkku & Telvis.
----------------------------------------------------------------------
Comment By: Stefan Becker (stefanb2)
Date: 2010-12-25 15:02
Message:
Courtesy of AirBerlin and Ms. Holle I spent plenty of hours on the
Helsinki-Vantaa airport and had time to progress the implementation a lot.
Over the last few days I finalized the rewrite and it should now support
all the features of the original grabber. If you want to have a peek, here
is the URL of the read-only git repository:
git clone http://chemobejk.dyndns.org/git/tv_grab_fi_rewrite.git
It currently supports the sources telkku.com, tvnyt.fi and yle.fi (finnish
& swedish [I hope this information is the same as svenska.yle.fi that
tv_grab_fi_sv uses]). Execute merge.pl to generate the old-style
one-script-grabber from the modular sources.
Including tvnyt.fi sources in test.conf currently still leads to test
failures. I haven't figured out yet how to handle the overlapping
information between the days so that it works correctly for a multiple-day
fetch and a single-day-then-merge fetch.
Hyvää Joulua!
----------------------------------------------------------------------
Comment By: Per Lundberg (perlundberg)
Date: 2010-12-12 23:05
Message:
Here's a thought: if/when you are implementing support for multiple
sources, why not merge in some of the code I've written in the
tv_grab_fi_sv grabber, to make it possible to grab both Swedish and Finnish
listings at once?
Since only some of the channels are available in Swedish, you have to
resort to use the tv_grab_combiner if you want to combine Swedish listings
for the channels where possible, with Finnish listings for the other
channels. If the tv_grab_fi could do it all... setting up this kind of
setup would be simplified a lot.
Please, just ask (perlun at gmail) if you have any questions regarding
the fi_sv grabber; you should be able to merge in code from there pretty
easily, I'd hope. (It uses the XML DOM parser to parse the HTML data, since
I find that preferable than setting up a lot of regexps.)
----------------------------------------------------------------------
Comment By: Karl Dietz (dekarl)
Date: 2010-12-12 19:34
Message:
I'd send a mail telling them that you'd like to make a program (as part of
xmltv) to use their welho tv guide on your (custom built) HTPC asking for
permission to do so and share the program.
----------------------------------------------------------------------
Comment By: Stefan Becker (stefanb2)
Date: 2010-12-12 17:42
Message:
@dekari: any suggestions how?
----------------------------------------------------------------------
Comment By: Karl Dietz (dekarl)
Date: 2010-12-12 14:06
Message:
@all: please, remember to get permission to use for each additional data
source.
@pkorpine: thanks for making your script API compatible
----------------------------------------------------------------------
Comment By: Ville Ahonen (va1210)
Date: 2010-12-12 11:18
Message:
Yes, this looks like a perfect addition to Stefan's rewrite of tv_grab_fi!
(And yes, "vapaa aika" == "recreational")
----------------------------------------------------------------------
Comment By: Stefan Becker (stefanb2)
Date: 2010-12-12 10:26
Message:
I'm currently working on a tv_grab_fi rewrite from scratch. One of the new
features I'm looking at is supporting multiple sources. I'll add tvnyt.fi
(or is it actually welho who provides this service???) to the list of
sources.
Thanks for the Python code. They have hidden getChannelPrograms.aspx very
well in the JavaScript code:-)
One good thing: the data contains a category which could be translated to
XMLTV information:
0 - unknown
1 - dokumentit (documentary)
2 - draama (drama)
3 - lapset (children)
4 - uutiset (news)
5 - urheilu (sports)
6 - vapaa aika (recreational?)
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=424138&aid=3135167&group_id=39046