Discussion:
[Xmltv-devel] Automatically notifying about SD-JSON lineup changes
John Veness
2016-07-06 13:07:12 UTC
Permalink
Hello,

I'm using tv_grab_sd_json, with selected channels from the
GBR-0001207-DEFAULT lineup (Oxford UK Freeview DVB-T transmissions). I
first set this up a week ago, and among the channels I picked was one
that the grabber called "Channel 5 Plus 24 (Free)" and which had ID
97294. All was fine for a few dyas.

Yesterday I noticed that I had no more listings for Channel 5+24. I
raised this with Schedules Direct. It turns out that they changed the
lineup for GBR-0001207-DEFAULT removing "Channel 5 Plus 24 (Free)" ID
97294 and adding "Channel 5 Plus 24" ID 56888.

(The difference is that the new channel has listings for 24 hours
whereas the old had large blocks of "SIGN OFF". I thought that SD had
made a mistake because as far as I was aware Channel 5+24 was not 24
hours on Freeview (and indeed
https://www.freeview.co.uk/why-freeview/channels/524 says it isn't 24
hours) but I notice from direct observation that SD are indeed correct
and that the official website is out of date!)
If the lineup changes upstream, then your grabber should realize that
(there's a field that indicates when the lineup was updated) and then
let you know that, or change the lineup, or something.
"metadata":{"lineup":"GBR-0001207-DEFAULT","modified":"2016-07-01T20:04:14Z","transport":"DVB-T"}}
So the question is, does tv_grab_sd_json "notice" these SD lineup
changes based on the modified date, and if not, can it be made to do so?
How would these be best communicated to downstream apps?

In my case I use mythfilldatabase to call tv_grab_sd_json. Would it be
appropriate to modify the MythTV database automatically to reflect the
lineup changes?

I realise this is a big problem, as in many case this would require
retuning, although not in this particular case. But if nothing else, it
would be good for the grabber (and mythfilldatabase) to inform the user
about the lineup change, based on the modified date.

What do people think?

John
Gary Buhrmaster
2016-07-06 18:40:22 UTC
Permalink
On Wed, Jul 6, 2016 at 1:07 PM, John Veness
Post by John Veness
I realise this is a big problem, as in many case this would require
retuning, although not in this particular case. But if nothing else, it
would be good for the grabber (and mythfilldatabase) to inform the user
about the lineup change, based on the modified date.
There are a couple of issues here, but for various
reasons, for some of my lineups (US based)
the lineups change quite regularly (at least for
some lineups), some of the changes seem to
be essentially "background", so you would
likely get too many notifications. For some
that would be goodness, for others annoyance.

It should be noted there is no current method that
mfdb has that would notify you of changes. It just
runs the grabber and takes the results. There is
nothing in the xml that supports a lineup last
updated date (only a date for when the xml was
generated), and nothing in the mythtv database
that stores that (so it could report something).
An xml dtd update would be needed to optionally
provide that additional information to mfdb, and
then you can submit patches to the mythtv project
to use it.

However, the base issue is that the channel changed
the xmltv id, for which there is currently no
automated way to reliably fix (or initially generate
the matches) although mfdb channel updates will
sometimes do so for some lineups/providers if
one does not disable channel updates (the attempts
at matching sometimes work for some changes
for some lineups).

I happen to believe that better long term (for the mfdb
case) is for mfdb to automate configuration and
detect changes via --get-lineup, and --list-channels
but that requires changes to mfdb to process that data.
AFAIK no one is currently working on that (it is probably
on some wish list).

As an interim solution (which you can do now),
outside of the mfdb usage cases, perhaps you
should save the output of the grabbers --list-channels
output and compare on a regular basis. If the
output has changed, you need to do some research.

Indirectly, for tv_grab_na_sd, the (non-quiet) output
displays when it is downloading updated lineups,
but that was never a primary purpose of the message.
Kevin Groeneveld
2016-07-06 23:28:50 UTC
Permalink
Post by John Veness
I'm using tv_grab_sd_json, with selected channels from the
GBR-0001207-DEFAULT lineup (Oxford UK Freeview DVB-T transmissions). I
first set this up a week ago, and among the channels I picked was one
that the grabber called "Channel 5 Plus 24 (Free)" and which had ID
97294. All was fine for a few dyas.
In addition to the good information that Gary already postedI thought I
would add one point:

If you configured tv_grab_sd_json to use the entire lineup instead of
selecting specific channels then you would always get all the channels in
the output even when the lineup changes. Depending how MythTV is
configured I think it can automatically add new channels to it's database
if they are present in the XMLTV output. Although I seem to recall having
issues with this when I last tried it. And if the lineup has a lot of crap
in it you really don't want this could be more annoying than your current
situation. Although I think you can hide channels in MythTV? Something to
consider anyways.


Kevin
Gary Buhrmaster
2016-07-07 00:06:10 UTC
Permalink
Post by Kevin Groeneveld
If you configured tv_grab_sd_json to use the entire lineup instead of
selecting specific channels then you would always get all the channels in
the output even when the lineup changes.
Excellent reminder. Yes, and that is the way I nominally
recommend using advanced grabbers with MythTV (see
below for a bit of info), and the way I do so. Note that
for grabbers that truly take significantly more resources
to pull additional channels this may not be the best
solution, but the Schedules Direct formats (shared
program data and grabber local caching) means that
there is far less "wasteful work" done by downloading
it all.
Post by Kevin Groeneveld
Depending how MythTV is configured
I think it can automatically add new channels to it's
database if they are present in the XMLTV output.
Although I seem to recall having issues with
this when I last tried it.
Yes, there are some "issues" there, and I have a few on my
(overly large) TODO list to address, but while I really want
to see the --get-lineup improvements, a more mundane
fix is far more likely to get done sooner (at least by me,
although we are still talking not real soon), and likely
easier to get accepted by the MythTV devs. On the
other hand, the mythtv project is always looking for a
some new devs, so if someone has an itch and wants
to scratch it, I am sure the MythTV devs want to talk.

In particular, I do not think MythTV will add channels
for scannable sources (at this time) because it
cannot properly create the entries (which requires
the network/service/program ids available with
--get-lineups). And there is a "feature" where
unscannable lineup channels can be added
again and again which is also on my TODO list.

mfdb does, however, try to match existing entries
based on some of the description and update the
first channel with the updated xmltvid. The matches
sometimes work, and they sometimes do not,
depending on many factors.
Post by Kevin Groeneveld
And if the lineup has a lot of crap in it you
really don't want this could be more annoying than your current situation.
Although I think you can hide channels in MythTV?
Yes, you can mark channels as "not visible"/"invisible"
(I forget the term on the page). That is often the easiest
solution (as with much else, your particular use cases
will vary). It is what I do for (about) 300 channels that
I do not subscribe to in my lineup. MythTV will fill
in the guide data (it is just a little more database space),
and then it will not (by default) use them to display
the guide or schedule recording. And it moves all
the control to one place (MythTV) rather than trying
to coordinate it two places. Rather than "deleting"
channels you are not interested in, or you cannot
really receive, one can mark them as "invisible".


As I suggested above, in this case, I am not sure if
MythTV's automatic updating would have resulted in
goodness in this case, so I am not going to say
it would have worked if only you did not filter the
list, but it would have been interesting if the OP tried
it (if nothing else, it would be an interesting data
point).
John Veness
2016-07-07 08:21:01 UTC
Permalink
Post by Gary Buhrmaster
Yes, you can mark channels as "not visible"/"invisible"
(I forget the term on the page). That is often the easiest
solution (as with much else, your particular use cases
will vary). It is what I do for (about) 300 channels that
I do not subscribe to in my lineup. MythTV will fill
in the guide data (it is just a little more database space),
and then it will not (by default) use them to display
the guide or schedule recording. And it moves all
the control to one place (MythTV) rather than trying
to coordinate it two places. Rather than "deleting"
channels you are not interested in, or you cannot
really receive, one can mark them as "invisible".
I don't do this at the moment, as I just replied to Kevin. I suppose I
could, but it feels conceptually "wrong" for so many channels to appear
in the GBR-0001207-DEFAULT lineup which shouldn't be there, and feels
wrong to use MythTV to fix this when the problem should be fixed at
source, in my opinion.

Incidentally, GBR-0001207-DEFAULT seems to have a description of "London
| DVB-T | Freeview - Central (South)", which sounds OK as I definitely
want DVB-T Freeview and Central (South) sounds about right, although the
London is a bit odd as I'm nowhere near London. But it definitely says
Freeview and Sky 1 etc. are not on Freeview. I realise this is a
Schedules Direct issue and not XMLTV.

John
--
John Veness, MythTV user, UK, DVB-T
Ian Campbell
2016-07-07 08:39:21 UTC
Permalink
Post by John Veness
Post by Gary Buhrmaster
Yes, you can mark channels as "not visible"/"invisible"
(I forget the term on the page).  That is often the easiest
solution (as with much else, your particular use cases
will vary).  It is what I do for (about) 300 channels that
I do not subscribe to in my lineup.  MythTV will fill
in the guide data (it is just a little more database space),
and then it will not (by default) use them to display
the guide or schedule recording.  And it moves all
the control to one place (MythTV) rather than trying
to coordinate it two places.  Rather than "deleting"
channels you are not interested in, or you cannot
really receive, one can mark them as "invisible".
I don't do this at the moment, as I just replied to Kevin. I suppose

could, but it feels conceptually "wrong" for so many channels to
appear 
in the GBR-0001207-DEFAULT lineup which shouldn't be there, and
feels 
wrong to use MythTV to fix this when the problem should be fixed at 
source, in my opinion.
Incidentally, GBR-0001207-DEFAULT seems to have a description of "London 
Post by Gary Buhrmaster
DVB-T | Freeview - Central (South)", which sounds OK as I definitely 
want DVB-T Freeview and Central (South) sounds about right, although the 
London is a bit odd as I'm nowhere near London. But it definitely says 
Freeview and Sky 1 etc. are not on Freeview. I realise this is a 
Schedules Direct issue and not XMLTV.
I reported this to SD after I switched and the answer from their
upstream was (paraphrasing) that Sky 1 etc are a component of the UK's
Digital Terrestrial TV platform which includes more that just the
obvious "Free" OTA channels but also contains pay IPTV channels
available from the likes of VuTV, BT or TalkTalk if you have
appropriate STB. All channels from 300-599 fit into that category.

Ian.
Robert Kulagowski
2016-07-07 16:12:26 UTC
Permalink
On Thu, Jul 7, 2016 at 3:21 AM, John Veness
Post by John Veness
Incidentally, GBR-0001207-DEFAULT seems to have a description of "London
| DVB-T | Freeview - Central (South)", which sounds OK as I definitely
want DVB-T Freeview and Central (South) sounds about right, although the
London is a bit odd as I'm nowhere near London. But it definitely says
Freeview and Sky 1 etc. are not on Freeview. I realise this is a
Schedules Direct issue and not XMLTV.
What would be a better location to put into the response? The
transmitter name? Something else?

(That's an easy enough fix to make on the Schedules Direct end if I
know what the community wants.)
John Veness
2016-07-07 18:29:38 UTC
Permalink
Post by Robert Kulagowski
What would be a better location to put into the response? The
transmitter name? Something else?
(That's an easy enough fix to make on the Schedules Direct end if I
know what the community wants.)
Well, it would be useful to know what the different fields separated by vertical bars are actually for. I'm not sure if the bars are generated at the SD end, or if they were added by the tv_grab_sd_json grabber.

If I look for lineups for postcode HP18 I get offered:

1. Oxford | Cable | Virgin Media Oxford - Digital | GBR-0000066-X
2. Watford | Cable | Virgin Media Watford - Cable | GBR-0000200-DEFAULT
3. Watford | Cable | Virgin Media Watford - Digital | GBR-0000200-X
4. United Kingdom | Satellite | BSkyB - England | GBR-0001172-DEFAULT
5. Luton | Cable | Virgin Media Milton Keynes - Cable | GBR-0001187-DEFAULT
6. Luton | Cable | Virgin Media Milton Keynes - Digital | GBR-0001187-X
7. United Kingdom | Satellite | BSkyB - Scotland | GBR-0001189-DEFAULT
8. United Kingdom | Satellite | BSkyB - Wales | GBR-0001190-DEFAULT
9. United Kingdom | Satellite | BSkyB - Northern Ireland | GBR-0001191-DEFAULT
10. United Kingdom | Satellite | BSkyB - Channel Isles | GBR-0001192-DEFAULT
11. United Kingdom | Satellite | BSkyB - Eire (Ireland) | GBR-0001193-DEFAULT
12. London | DVB-T | Freeview - Central (South) | GBR-0001207-DEFAULT
13. London | DVB-T | Freeview - Anglia/Central (South) | GBR-0001311-DEFAULT
14. United Kingdom | Satellite | BSkyB - United Kingdom | GBR-0001505-DEFAULT
15. United Kingdom | Satellite | Freesat - Channel Isles | GBR-0003011-DEFAULT
16. United Kingdom | Satellite | Freesat - England | GBR-0003012-DEFAULT
17. United Kingdom | Satellite | Freesat - Northern Ireland | GBR-0003013-DEFAULT
18. United Kingdom | Satellite | Freesat - Scotland | GBR-0003014-DEFAULT
19. United Kingdom | Satellite | Freesat - Wales | GBR-0003015-DEFAULT

John
Gary Buhrmaster
2016-07-07 18:45:14 UTC
Permalink
On Thu, Jul 7, 2016 at 6:29 PM, John Veness
Post by John Veness
Well, it would be useful to know what the different fields separated by vertical bars are actually for. I'm not sure if the bars are generated at the SD end, or if they were added by the tv_grab_sd_json grabber.
The bars are not in the raw data, but are a display artifact
in the app/grabber. The SD data (typically) contains
a location, a transport (i.e. cable, antenna, satellite), a name,
and the actual lineup (which can be displayed in any order
by the app).

I believe most of that data starts from "upstream",
and has varying degrees of helpfulness and
accuracy. A name is just a name, and a rose
by any other name... I always look at the
information as a hint, rather than treat it as
have any specific accuracy.
Robert Kulagowski
2016-07-07 19:09:37 UTC
Permalink
On Thu, Jul 7, 2016 at 1:45 PM, Gary Buhrmaster
Post by Gary Buhrmaster
I believe most of that data starts from "upstream",
and has varying degrees of helpfulness and
accuracy. A name is just a name, and a rose
by any other name... I always look at the
information as a hint, rather than treat it as
have any specific accuracy.
Yes, most of the data starts at Gracenote. I make local changes to
normalize things, and remove things that are incorrect. I can change
the location of "London" on Freeview but would need to know what's
more appropriate for each lineup.
Hika van den Hoven
2016-07-07 20:57:41 UTC
Permalink
Hoi Robert,
Post by Robert Kulagowski
On Thu, Jul 7, 2016 at 1:45 PM, Gary Buhrmaster
Post by Gary Buhrmaster
I believe most of that data starts from "upstream",
and has varying degrees of helpfulness and
accuracy. A name is just a name, and a rose
by any other name... I always look at the
information as a hint, rather than treat it as
have any specific accuracy.
Yes, most of the data starts at Gracenote. I make local changes to
normalize things, and remove things that are incorrect. I can change
the location of "London" on Freeview but would need to know what's
more appropriate for each lineup.
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
xmltv-devel mailing list
https://lists.sourceforge.net/lists/listinfo/xmltv-devel
Bars is what for instance thetvdb uses as list separator. A proper
graber would translate that back to a list.

Tot mails,
Hika Alina Maria van den Hoven mailto:***@gmail.com

Look at https://github.com/tvgrabbers/tvgrabnlpy for my Python
tvgrabAPI and at https://github.com/tvgrabbers/DataTree for its
kernel.

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens
--
Robert Kulagowski
2016-07-07 21:15:15 UTC
Permalink
Post by Hika van den Hoven
Bars is what for instance thetvdb uses as list separator. A proper
graber would translate that back to a list.
The data is sent in JSON without bars at all. If there are bars then
it's something that the grabber is doing when it takes the JSON data
and puts it on the screen.
Hika van den Hoven
2016-07-07 21:30:31 UTC
Permalink
Hoi Robert,
Post by Robert Kulagowski
Post by Hika van den Hoven
Bars is what for instance thetvdb uses as list separator. A proper
graber would translate that back to a list.
The data is sent in JSON without bars at all. If there are bars then
it's something that the grabber is doing when it takes the JSON data
and puts it on the screen.
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
xmltv-devel mailing list
https://lists.sourceforge.net/lists/listinfo/xmltv-devel
OK so this grabber uses it to denote a list on screen? So at least it
is bad formatting and probably no formatting at all using system
standards, creating confusion.

Tot mails,
Hika Alina Maria van den Hoven mailto:***@gmail.com

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens
--
Kevin Groeneveld
2016-07-08 00:14:21 UTC
Permalink
Post by Hika van den Hoven
Hoi Robert,
Post by Robert Kulagowski
Post by Hika van den Hoven
Bars is what for instance thetvdb uses as list separator. A proper
graber would translate that back to a list.
The data is sent in JSON without bars at all. If there are bars then
it's something that the grabber is doing when it takes the JSON data
and puts it on the screen.
OK so this grabber uses it to denote a list on screen? So at least it
is bad formatting and probably no formatting at all using system
standards, creating confusion.
The bars aren't really denoting a list but separating different fields of
information for one lineup. This is a common way of separating fields in
text output from database queries such as from SQLite or MySQL. Maybe not
the best choice in this case but personally I don't really see what the
confusion is. Do you have a suggestion for a better way to show it?


Kevin
John Veness
2016-07-08 08:43:25 UTC
Permalink
Post by Robert Kulagowski
On Thu, Jul 7, 2016 at 1:45 PM, Gary Buhrmaster
Post by Gary Buhrmaster
I believe most of that data starts from "upstream",
and has varying degrees of helpfulness and
accuracy. A name is just a name, and a rose
by any other name... I always look at the
information as a hint, rather than treat it as
have any specific accuracy.
Yes, most of the data starts at Gracenote. I make local changes to
normalize things, and remove things that are incorrect. I can change
the location of "London" on Freeview but would need to know what's
more appropriate for each lineup.
I didn't mean to start a big debate about the bars - I was only pointing
them out as I thought that Robert K might wonder where they came from
(the tv_grab_sd_json grabber) if they didn't come from SD.

As to what each lineup should be called, I'm no expert, so maybe someone
else who knows a lot about UK TV can chime in. I notice that the list of
lineups you have on
https://json.schedulesdirect.org/20141201/transmitters/GBR have names
that seem to correspond with "Transmitter regions" as listed on
https://ukfree.tv/maps/freeviewlight for example.
http://help.digitaluk.co.uk/pf/197/webfiles/mux_list.pdf also seems to
give a list (in small print) of transmitter regions, and says what
channels are available in which region, although it doesn't go into
detail about the different ITV regions (local news etc.)

John
--
John Veness, MythTV user, UK, DVB-T
Robert Kulagowski
2016-07-08 16:09:40 UTC
Permalink
On Fri, Jul 8, 2016 at 3:43 AM, John Veness
Post by John Veness
As to what each lineup should be called, I'm no expert, so maybe someone
else who knows a lot about UK TV can chime in. I notice that the list of
lineups you have on
https://json.schedulesdirect.org/20141201/transmitters/GBR have names
that seem to correspond with "Transmitter regions" as listed on
https://ukfree.tv/maps/freeviewlight for example.
http://help.digitaluk.co.uk/pf/197/webfiles/mux_list.pdf also seems to
give a list (in small print) of transmitter regions, and says what
channels are available in which region, although it doesn't go into
detail about the different ITV regions (local news etc.)
If "you guys" can come up with what it should be because it makes
sense to you (since you're directly affected) then it's a simple fix
on my side. I just want the community to come to a consensus.
Jan Ceuleers
2016-07-08 17:35:50 UTC
Permalink
Post by Robert Kulagowski
If "you guys" can come up with what it should be because it makes
sense to you (since you're directly affected) then it's a simple fix
on my side. I just want the community to come to a consensus.
Sorry if this muddies the water, but let me add the perspective of a
user of UK listings data who is located outside the UK. I receive a
number of UK channels via my Belgian cable provider, and I'm using the
SD-JSON grabber for listings data mostly because the listings data
provided locally (hi Hika) is in Dutch whereas I prefer the original
English.

Whereas I understand that SD's concept of lineups is intended to be
helpful, it's actually a complication for me. I'd welcome a way to
configure the grabber that cuts through the lineup and that gets me
straight to the channels. Not that I need it right now since I have it
set up using a lineup for a service area that I'm not located within,
but I hope that this will not prevent me in the future from accessing
listings data for other European countries as it becomes available in SD.

Thanks, Jan
Gary Buhrmaster
2016-07-08 21:46:49 UTC
Permalink
Post by Jan Ceuleers
Whereas I understand that SD's concept of lineups is intended to be
helpful, it's actually a complication for me. I'd welcome a way to
configure the grabber that cuts through the lineup and that gets me
straight to the channels.
I have no affiliation with Schedules Direct (other than as
a "member"/"customer"), but I can see a number of
good, bad, and ugly reasons that the channels are
assigned to lineups, and you can only have 4 lineups
in your account (which are expected to be "reasonably"
locality associated). And the most obvious ugly reason
is that some individuals would decided to add every
single channel in the world to their "pull", and resell
the data (all for the low low price of $25/yr(*)). It is likely
some people are already "sharing" their account data
with non-individual affiliated persons. Yes, there are people
who think that way, and act that way. And that is why
we can not have nice things.

Schedules Direct has been reasonably responsive if
there is a missing "lineup" (well, at least for countries
that they serve). And while Belgium is not yet a country
served, they may have some ability to add that if there
is interest. As a customer (or potential customer), you
might want to ask them directly if they can provide
Belgium cable data (Yes, many SD people read this list,
and may response, but SD specific requests probably
should really be performed directly with SD, which has
an entire process to make sure requests get addressed
and responded to).

Gary



(*) Funny thing is that writing that sentence reminded
me that it was time to renew my membership.
Jan Ceuleers
2016-07-09 06:27:06 UTC
Permalink
Post by Gary Buhrmaster
Schedules Direct has been reasonably responsive if
there is a missing "lineup" (well, at least for countries
that they serve). And while Belgium is not yet a country
served, they may have some ability to add that if there
is interest. As a customer (or potential customer), you
might want to ask them directly if they can provide
Belgium cable data
I'm afraid that this wouldn't solve all of my problem, because most
people located in Belgium would probably want the guide data to be in
their own language (whereas I prefer it to be either in English or in
the channel's original language).

Here in Belgium there are three official languages (Dutch, French and
German), and there are Belgian channels in each of these languages. In
addition we of course also get channels from lots of other countries in
Europe and further afield. The guide data on the cable STB is shown in
whatever the user interface language is set to (Dutch or French) even
for Italian/Spanish/etc channels. So someone/somewhere is translating
guide data.

So the concept of a lineup would need to be enriched with a preferred UI
language (or perhaps a prioritised list of languages).

Another aspect: policies concerning treatment of foreign-language films
and their impact on duplicate detection. If a Dutch-language channel
puts out an English-language movie it will announce the original
(English) title, it will put out the original (English) soundtrack and
it will burn Dutch subtitles into the video. However, channels whose
original language is e.g. German or French will translate the title in
the guide data, put out the original video but a translated (dubbed)
soundtrack. Duplicate checking across these policies is not currently
possible. This problem is of course not SD-specific. But whether the end
user would regard "Die Unbestechlichen" as a duplicate of "The
Untouchables" is a matter of local policy.
Post by Gary Buhrmaster
(Yes, many SD people read this list,
and may response, but SD specific requests probably
should really be performed directly with SD, which has
an entire process to make sure requests get addressed
and responded to).
I might do that, but I'd be interested in views in this forum as to how
much of the problem domain can/should be tackled in the XMLTV project
versus upstream (i.e. data sources) or downstream (i.e. consumers of
guide data such as MythTV).

Thx, Jan
Hika van den Hoven
2016-07-09 07:06:00 UTC
Permalink
Hoi Jan,

Saturday, July 9, 2016, 8:27:06 AM, you wrote:

...
Post by Jan Ceuleers
Another aspect: policies concerning treatment of foreign-language films
and their impact on duplicate detection. If a Dutch-language channel
puts out an English-language movie it will announce the original
(English) title, it will put out the original (English) soundtrack and
it will burn Dutch subtitles into the video. However, channels whose
original language is e.g. German or French will translate the title in
the guide data, put out the original video but a translated (dubbed)
soundtrack. Duplicate checking across these policies is not currently
possible. This problem is of course not SD-specific. But whether the end
user would regard "Die Unbestechlichen" as a duplicate of "The
Untouchables" is a matter of local policy.
Interesting. You're right German and French channels often have
translated titles, while Dutch and Belgian not. xmltv however allows
to add multiple titles in multiple languages. I do not know how Mythtv
and others deal with this. Once I have my new API stable I can see if
I through ttvdb and tmdb could add options in that respect. I already
if encountered do add an "original title".
Post by Jan Ceuleers
Thx, Jan
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
xmltv-devel mailing list
https://lists.sourceforge.net/lists/listinfo/xmltv-devel
Tot mails,
Hika Alina Maria van den Hoven mailto:***@gmail.com

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens
--
Gary Buhrmaster
2016-07-10 01:22:09 UTC
Permalink
Post by Jan Ceuleers
This problem is of course not SD-specific. But whether the end
user would regard "Die Unbestechlichen" as a duplicate of "The
Untouchables" is a matter of local policy.
I have no idea what the raw data looks like for that
example (is it provided as the same SHow ID, or
different SHow IDs) but at least some description fields
include a language format available in the json
source data. And (as I recall) the xmltv definition
allows multiple titles in different languages, but
a bunch do not.

There has been a proposal over in the SD forums
to provide the ability for an app to get the alternative
programs (and titles) in other languages for a
program. This will provide a possible way forward.

So, at least in theory, you can actually have both titles
in the xmltv data, and then the app (in this case MythTV)
could decide which to display to you. Of course
(last I knew) MythTV does not store both (pushing
the problem to the app, but that is likely where it
needs to be anyway, since the app might be
supporting dual language households (where one
person wants (say) Spanish, and another wants
(say) English).

But, at least at this point, this is a work in progress
(on many fronts).
Robert Kulagowski
2016-07-11 15:48:28 UTC
Permalink
Post by Robert Kulagowski
On Fri, Jul 8, 2016 at 3:43 AM, John Veness
Post by John Veness
As to what each lineup should be called, I'm no expert, so maybe someone
else who knows a lot about UK TV can chime in. I notice that the list of
lineups you have on
https://json.schedulesdirect.org/20141201/transmitters/GBR have names
that seem to correspond with "Transmitter regions" as listed on
https://ukfree.tv/maps/freeviewlight for example.
http://help.digitaluk.co.uk/pf/197/webfiles/mux_list.pdf also seems to
give a list (in small print) of transmitter regions, and says what
channels are available in which region, although it doesn't go into
detail about the different ITV regions (local news etc.)
If "you guys" can come up with what it should be because it makes
sense to you (since you're directly affected) then it's a simple fix
on my side. I just want the community to come to a consensus.
I've created a thread at

http://forums.schedulesdirect.org/viewforum.php?f=16

to solicit opinions from users that are in the regions.

John Veness
2016-07-07 08:11:16 UTC
Permalink
On Wed, Jul 6, 2016 at 2:40 PM, Gary Buhrmaster
Post by John Veness
I'm using tv_grab_sd_json, with selected channels from the
GBR-0001207-DEFAULT lineup (Oxford UK Freeview DVB-T transmissions). I
first set this up a week ago, and among the channels I picked was one
that the grabber called "Channel 5 Plus 24 (Free)" and which had ID
97294. All was fine for a few dyas.
In addition to the good information that Gary already postedI thought
If you configured tv_grab_sd_json to use the entire lineup instead of
selecting specific channels then you would always get all the channels
in the output even when the lineup changes. Depending how MythTV is
configured I think it can automatically add new channels to it's
database if they are present in the XMLTV output. Although I seem to
recall having issues with this when I last tried it. And if the
lineup has a lot of crap in it you really don't want this could be
more annoying than your current situation. Although I think you can
hide channels in MythTV? Something to consider anyways.
Kevin
Thanks Kevin. Unfortunately the entire lineup (which for me, according
to https://json.schedulesdirect.org/20141201/transmitters/GBR picking
the Oxford transmitter, is GBR-0001207-DEFAULT) seems to contain 80+
channels which are not actually transmitted from that transmitter,
including Sky 1, Sky Movies, Comedy Central. I don't know why this is. I
have reported this to Schedules Direct and Robert K said that when he
"Freeview" represents UK's digital terrestrial television (DTT). The FULL range of channels broadcast via UK's DTT includes free, subscription and pay television services from operators such as VuTV and BT. All the channels from 300 to 599 are optional IPTV channels - available only if the customer is using a single set-top-box for Freeview and an additional service provider like VuTV, BT or TalkTalk. It is still one box.
"Freeview" is the name of the platform and NOT a statement that all channels are "free". The end user may be only focusing on the "free" portion and not the FULL line-up.
That personally doesn't make sense to me. Freeview is, as far as I know,
95%+ of channels broadcast on DVB-T/DVB-T2 in UK. There are a handful of
encrypted channels on DVB-T which as far as I'm concerned are not
Freeview - the "famous" provider (I think now defunct) used to be called
Top-Up TV, and there was a bit of a grey area in the general population
between Freeview and Top-Up TV, although as I say, I would personally
not include the Top-Up TV channels as part of Freeview.

But the IPTV (i.e. streaming over the Internet) channels they mention
are nothing to do with Freeview as far as I'm concerned - just because
one box can show Freeview and streaming doesn't make streaming part of
Freeview or DVB-T. They are completely different platforms.

It's not a massive problem, as I can say "no" to the channels I don't
want, but it is a potential source of confusion and is, I think,
basically "wrong" to include all the channels I mentioned as if they
were broadcast from the Oxford transmitter or any other DVB-T/DVB-T2
transmitter.

John
--
John Veness, MythTV user, UK, DVB-T
Loading...