[Xmltv-devel] [xmltv-commit] xmltv/grab/eu_epgdata tv_grab_eu_epgdata, 1.35, 1.36

[Karl Dietz]

! Examples: "" (to activate automatic detection), "local" (your local system timezone), "Europe/Berlin", "+0100" (without quotation marks)',
+ if (not $tz) {
+ warn "Unable to determine country/timezone of data. ",
+ "You may specify tz in your configuration. ",
+ "Falling back to your local system time zone.\n";
+ $tz = 'local';
+ }

just a heads up, "local" time *must* have an explicit offset to be
valid. With the upcoming 0.26 release of MythTV floating local time
will stop working!

Regards,
Karl

[Jan Schneider]

Post by Karl Dietz

! Examples: "" (to activate automatic detection), "local"
(your local system timezone), "Europe/Berlin", "+0100" (without
quotation marks)',
+ if (not $tz) {
+ warn "Unable to determine country/timezone of data. ",
+ "You may specify tz in your configuration. ",
+ "Falling back to your local system time zone.\n";
+ $tz = 'local';
+ }

just a heads up, "local" time *must* have an explicit offset to be
valid. With the upcoming 0.26 release of MythTV floating local time
will stop working!

So I guess the best approach is, now that we have automatic TZ
detection in the grabber, to completely drop the "local" option after
testing the new detection in the next XMLTV release?

[Robert Eden]

Post by Karl Dietz
just a heads up, "local" time *must* have an explicit offset to be
valid. With the upcoming 0.26 release of MythTV floating local time
will stop working!

Karl, is that a violation of the XMLTV DTD? IIRC you can display as
much or as little of a time as you know.

Robert

[Karl Dietz]

Post by Robert Eden

Post by Karl Dietz
just a heads up, "local" time *must* have an explicit offset to be
valid. With the upcoming 0.26 release of MythTV floating local time
will stop working!

Karl, is that a violation of the XMLTV DTD? IIRC you can display as
much or as little of a time as you know.

Yes, you can give as much or as little as you know. But you must be
explicit about the time offset/zone. Implicit its specified as UTC.
MythTV was defaulting to ignore the time offset and take everything as
local time. Thats going to change for the next release.

The spec does allow timezones instead of simple explicit offset, too.
But I would strongly advise against using that. (the spec mentions BST
as a valid example which is the abbreviation for 5 different offsets,
see http://en.wikipedia.org/wiki/BST )

Regards,
Karl

[Ben Bucksch]

Post by Karl Dietz
MythTV was defaulting to ignore the time offset and take everything as
local time. Thats going to change for the next release.

Getting the timezone right is excessively hard for grabbers.
I suggest to relax this in MythTV again (e.g. revert whatever change
made this more strict) and accept local time.

Ben

[Karl Dietz]

Post by Ben Bucksch

Post by Karl Dietz
MythTV was defaulting to ignore the time offset and take everything as
local time. Thats going to change for the next release.

Getting the timezone right is excessively hard for grabbers.

Why? The grabber is the only part of the pipeline where we can easily
convert floating time into proper time as only the grabber knows what
its getting from the data source.
That's the whole point of the grabber. To abstract the various quirks,
formats and additional data that is needed to use a guide source from
the consuming application.
Getting the timezone right is super easy for all but one hour per year.
Basically its just one call to "take this local time in Europe/Berlin
and give me the offset".
The only time when its hard is when localtime overlaps. In the german
case its the distinction between 2 o'clock and 2a o'clock when we
switch back from summer to winter time.

Post by Ben Bucksch
I suggest to relax this in MythTV again (e.g. revert whatever change
made this more strict) and accept local time.

In what case would it be good to move back to known broken floating
local time?

Regards,
Karl

PS: For an example see tv_grab_eu_epgdata that was converted from
varying floating local times to proper automatically determined
offsets lately.
http://xmltv.cvs.sf.net/viewvc/xmltv/xmltv/grab/eu_epgdata/tv_grab_eu_epgdata?r1=1.35&r2=1.36

[Ben Bucksch]

Post by Karl Dietz
Why? The grabber is the only part of the pipeline where we can easily
convert floating time into proper time as only the grabber knows what
its getting from the data source.

Because of summer time, basically.
Usually, the source gives no timezone and assumes "local time".
Now, even though I know it's always German time, I don't know whether
it's +0100 or +0200. And the boundary to summer time is typically hard
or impossible to calculate.

It's true that Linux has time zone info with usually proper summer time
boundaries, and some programming languages have Date tools that can
convert any arbitrary local time into a proper timezone.

But not all do. For example, if your data source is XML and you decide
to use XSLT to transform to XMLTV, then you generally have a problem.
(Esp. if you don't want to dictate certain XSLT implementations.)

The timezone problem is therefore a significant blocker for grabber
writers. If you mandate a full timezone, some grabber writers will give
up at the point where they realize that they ran into a dead end with
their programming environment.

Ben

[Karl Dietz]

Post by Ben Bucksch

Post by Karl Dietz
Why? The grabber is the only part of the pipeline where we can easily
convert floating time into proper time as only the grabber knows what
its getting from the data source.

Because of summer time, basically.

another point for "summer time must die" :)

...

Post by Ben Bucksch
The timezone problem is therefore a significant blocker for grabber
writers. If you mandate a full timezone, some grabber writers will give
up at the point where they realize that they ran into a dead end with
their programming environment.

I do not agree with that. If someone chooses to use a tool that is so
limiting with its options then they have to come up with many more work
arounds and the manual switch between summer / normal time should be
the least of their issues.

BTW this is not a new requirement, its been documented in the schema
since November 2001, we are just enforcing what has been mandatory for
more then 10 years because it allows to reduce complexity for the end
users.

If it helps I can write a filter to fix xmltv files in floating local
time by giving it the files and a proper time zone.
That way everyone can use whatever tools they like and still supply
consuming applications with proper data.

Would that work for you?

Regards,
Karl

[Ben Bucksch]

If someone chooses to use a tool that is so limiting with its options

This starts being political, "under the belt" in fact. XSLT is a tool
very much made for the task at hand (converting from and to XML), so
it's a fair choice to make.

Summer time, OTOH, can *only* be determined with a big database
reflecting laws and law changes and local circumstances around the
world. Is it normal that most tools are not up for the job. Timezone
conversion: yes. Converting current local time with known timezone into
another timezone: yes. But determining summer time for an *arbitrary*
point in time is highly difficult.

If it helps I can write a filter to fix xmltv files in floating local
time by giving it the files and a proper time zone.
That way everyone can use whatever tools they like and still supply
consuming applications with proper data.
Would that work for you?

This is mostly needed for third-party XSLT-based grabbers.
That filter would work standalone for grabbers that are not part of the
XMLTV distribution, and can be used independently from XMLTV project? If
so, that would work for me.
MythTV should then automatically invoke that tool when it finds no timezone.

Ben

[Karl Dietz]

Post by Ben Bucksch

If someone chooses to use a tool that is so limiting with its options

This starts being political, "under the belt" in fact. XSLT is a tool
very much made for the task at hand (converting from and to XML), so
it's a fair choice to make.

sorry, that was not my intention. But I having difficulties
understanding why you insist in making MythTV/XMLTV more complex then
it need be when the only thing to be gained is support for a use case
that I'm not aware of anyone using.

...

Post by Ben Bucksch

If it helps I can write a filter to fix xmltv files in floating local
time by giving it the files and a proper time zone.
That way everyone can use whatever tools they like and still supply
consuming applications with proper data.
Would that work for you?

This is mostly needed for third-party XSLT-based grabbers.

I'm not aware of any maintained third-party grabbers based on XSLT.
Our tv_grab_is uses XSLT but does not have that issue as Iceland's
local time matches UTC.
Can you give me an example? Maybe I can work with the maintainer to find
better solution.

Post by Ben Bucksch
That filter would work standalone for grabbers that are not part of the
XMLTV distribution, and can be used independently from XMLTV project? If
so, that would work for me.

Yes and no. My intention was to offer adding a new filter to the XMLTV
project that can be used on any source file.
But I do not intent to make it independent of the distribution. So it
would be part of the prebuilt windows executable. I'm not interested
in writing a portable application from scratch and setup proper
distribution.

Post by Ben Bucksch
MythTV should then automatically invoke that tool when it finds no timezone.

I don't see the point in breaking proper xmltv files (with implicit
UTC) in favor of supporting broken xmltv files (with floating local
time).

Regards,
Karl

[Ben Bucksch]

Post by Karl Dietz
understanding why you insist in making MythTV/XMLTV more complex then
it need be

Well, it depends on the amount of complexity. If it's just a few lines
of code in a "suitable programming environment" like MythTV has, then I
think it makes sense to keep it. If it's difficult and/or indeed there's
only my grabber with that problem, then I agree with you.

Post by Karl Dietz
I don't see the point in breaking proper xmltv files (with implicit
UTC) in favor of supporting broken xmltv files (with floating local
time).

Just for your information, opposite to what the spec says, from what I
know, grabbers have always meant local time when they omitted timezone,
and applications have interpreted it like that.

Post by Karl Dietz
I'm not aware of any maintained third-party grabbers based on XSLT.

I have one :-) (it's not public, because the source objected). I thought
I had seen others, too.

My grabber alone isn't enough reason, I concur.

Ben

[Karl Dietz]

Hi Ben,

I totally forgot about your private grabber, but now that you mention
it :)
Find a new filter in the repository, it works as follows.

Regards,
Karl

[***@virunas ~/xmltv]$ cat in.xml | tv_augment_tz
--tz=Europe/Berlin > out.xml
[***@virunas ~/xmltv]$ diff -ruN in.xml out.xml
--- in.xml 2012-06-13 08:32:34.498671256 +0200
+++ out.xml 2012-06-13 08:32:38.844372386 +0200
@@ -4,16 +4,16 @@
<channel id="I34156.labs.zap2it.com">
<display-name>TestChannel</display-name>
</channel>
- <programme start="20111030000000 +0000" stop="20111030003000 +0000"
channel="I34156.labs.zap2it.com">
+ <programme start="20111030020000 +0200" stop="20111030023000 +0200"
channel="I34156.labs.zap2it.com">
<title lang="en">Rachel's Favorite Food at Home</title>
</programme>
- <programme start="20111030003000 +0000" stop="20111030010000 +0000"
channel="I34156.labs.zap2it.com">
+ <programme start="20111030023000 +0200" stop="20111030020000 +0100"
channel="I34156.labs.zap2it.com">
<title lang="en">Katie Brown Workshop</title>
</programme>
- <programme start="20111030010000 +0000" stop="20111030013000 +0000"
channel="I34156.labs.zap2it.com">
+ <programme start="20111030020000 +0100" stop="20111030023000 +0100"
channel="I34156.labs.zap2it.com">
<title lang="en">Endless Feast</title>
</programme>
- <programme start="201110300100" stop="201110300130"
channel="I34156.labs.zap2it.com">
+ <programme start="20111030010000 +0200" stop="20111030013000 +0200"
channel="I34156.labs.zap2it.com">
<title lang="en">floating local time</title>
</programme>
</tv>
[***@virunas ~/xmltv]$

[Ben Bucksch]

Post by Karl Dietz
Find a new filter in the repository, it works as follows.
tv_augment_tz

Thanks!

[Karl Dietz]

Hi,

I'm sending this message again as it has not made it back to me or into
the archives, yet. I hope you don't get it twice.

Regards,
Karl

-------- Original Message --------
Subject: Re: [Xmltv-devel] [xmltv-commit] xmltv/grab/eu_epgdata
tv_grab_eu_epgdata, 1.35, 1.36
Date: Sun, 10 Jun 2012 09:24:18 +0200

Post by Ben Bucksch

Post by Karl Dietz
MythTV was defaulting to ignore the time offset and take everything as
local time. Thats going to change for the next release.

Getting the timezone right is excessively hard for grabbers.

Why? The grabber is the only part of the pipeline where we can easily
convert floating time into proper time as only the grabber knows what
its getting from the data source.
That's the whole point of the grabber. To abstract the various quirks,
formats and additional data that is needed to use a guide source from
the consuming application.
Getting the timezone right is super easy for all but one hour per year.
Basically its just one call to "take this local time in Europe/Berlin
and give me the offset".
The only time when its hard is when localtime overlaps. In the german
case its the distinction between 2 o'clock and 2a o'clock when we
switch back from summer to winter time.

Post by Ben Bucksch
I suggest to relax this in MythTV again (e.g. revert whatever change
made this more strict) and accept local time.

In what case would it be good to move back to known broken floating
local time?

Regards,
Karl

PS: For an example see tv_grab_eu_epgdata that was converted from
varying floating local times to proper automatically determined
offsets lately.
http://xmltv.cvs.sf.net/viewvc/xmltv/xmltv/grab/eu_epgdata/tv_grab_eu_epgdata?r1=1.35&r2=1.36