[Xmltv-devel] --config-file option

[Max Barry]

Hello! I have a question about the --config-file option as presented here:

http://wiki.xmltv.org/index.php?title=XmltvCapabilities#manualconfig

... and here:

http://wiki.xmltv.org/index.php/HowtoUseGrabbers#Configure_the_grabber

In particular, I'm curious about why the latter instructs users: "It is
recommended that you always use this parameter." It follows with an
example command: "tv_grab_au --configure --config-file
/home/myuser/.myapp/tv_grab_au.conf".

It seems this makes things unnecessarily complicated for users, given
the XMLTV standard doesn't specify what a grabber should do in the
absence of this parameter (right?). The three possibilities I see:

(A) The grabber refuses to run (e.g. it says, "Missing required option:
--config-file").

(B) The grabber assumes a default config file and always ignores the
--config-file option. If I'm reading XmltvCapabilities right, this is
only legal behavior when the grabber does NOT claim the "manualconfig"
capability, i.e. doesn't support --configure either. If it supports
--configure, though, it must also support --config-file.

(C) The grabber assumes a default config file but allows this to be
overriden by --config-file.

And all of these seem to be valid behaviors, i.e. comply with
XmltvCapabilities.

So, if it's (A), it doesn't really matter what the wiki says. A user who
omits the parameter will be presented with an obvious problem. If
they're configuring for the first time, they will see that they have to
use --config-file; otherwise, they have to try to remember what they
used when they configured.

If it's (B), it doesn't matter much either, although it's kind of a
time-waster, telling users to employ a completely ignored --config-file
parameter. It might create confusion, too, should they go looking for
that file and not find it. And I guess some grabbers will refuse to run
if they're sent an option they don't support. Presumably the user
shouldn't be running --configure at all on these grabbers, but the wiki
page doesn't say that. (In fact, it says the opposite: "Before a grabber
can download data, it needs to be configured.")

If it's (C), life will be simple if the user didn't supply --config-file
when they first setup, but complicated if they did. In that case, the
grabber will now assume a different default config file and may produce
misleading output, such as saying it hasn't been configured yet! Then
the user is left to ponder what's going on.

In summary, the wiki recommendation seems to save some users from a very
minor problem -- omitting a parameter that some grabbers may require --
while potentially creating a murkier problem for everyone else.

Max.

P.S. Somewhat relatedly, I'm curious as to why a grabber MUST support
--config-file if it supports --configure. I know some users like to set
up multiple profiles, sometimes running a grabber with one config file,
sometimes with another, but surely that's fairly advanced behavior...
why must it be mandatory to support? It seems to encourage grabber devs
to simply shift the problem to users and always require them to remember
the right --config-file.

[Robert Eden]

Post by Max Barry
http://wiki.xmltv.org/index.php?title=XmltvCapabilities#manualconfig
http://wiki.xmltv.org/index.php/HowtoUseGrabbers#Configure_the_grabber
In particular, I'm curious about why the latter instructs users: "It is
recommended that you always use this parameter." It follows with an
example command: "tv_grab_au --configure --config-file
/home/myuser/.myapp/tv_grab_au.conf".

...

Post by Max Barry
(C) The grabber assumes a default config file but allows this to be
overriden by --config-file.

This is my experience. It's probably important for a "capabilities"
based config so the configuring program can modify the right config
file. If it's done by a human, the default is fine, unless you want
multiple config files. I think the default is ~/.xmltv/program.conf

Robert

[Karl Dietz]

Post by Max Barry
P.S. Somewhat relatedly, I'm curious as to why a grabber MUST support
--config-file if it supports --configure. I know some users like to set
up multiple profiles, sometimes running a grabber with one config file,
sometimes with another, but surely that's fairly advanced behavior...
why must it be mandatory to support? It seems to encourage grabber devs
to simply shift the problem to users and always require them to remember
the right --config-file.

Please keep in mind, that grabbers are also called by programs. And that
programs running as service may not be allowed to write to arbitrary
locations on the filesystem.

Regards,
Karl

[Max Barry]

Thank you for the replies!

Post by Karl Dietz

Post by Max Barry
P.S. Somewhat relatedly, I'm curious as to why a grabber MUST support
--config-file if it supports --configure. I know some users like to set
up multiple profiles, sometimes running a grabber with one config file,
sometimes with another, but surely that's fairly advanced behavior...
why must it be mandatory to support? It seems to encourage grabber devs
to simply shift the problem to users and always require them to remember
the right --config-file.

Please keep in mind, that grabbers are also called by programs. And that
programs running as service may not be allowed to write to arbitrary
locations on the filesystem.

This makes sense by itself, but highlights the problem I'm having with
the wiki. If a user follows its directions, and runs "tv_grab_au
--configure --config-file /home/myuser/.myapp/tv_grab_au.conf", and
tv_grab_au is subsequently called by a program... that program won't
know which --config-file the user chose.

So, at best, the user's time has just been wasted, as the program will
make the user re-configure. At worst, the grabber will run differently
depending on whether it's being run by the program or the user, which
will create some pretty confusing situations should that user attempt to
figure out what's going on.

So I see that a program might want to specify a --config-file that it
knows it can write to, and may even have a good reason for maintaining
its own config file, separate from whatever the user has setup. What I
don't understand is why the wiki makes this situation the default case,
by telling users to specify --config-file even when they don't need to,
and unnecessarily inviting the potential usability problems of dealing
with multiple config files.

Post by Karl Dietz
If a grabber requires a config file (and not all of them do) then it's reasonable to allow the user to change its file location.

I agree that it's a handy feature. But it has a downside in that it
allows users to accidentally end up with multiple config files. So it's
a good optional feature but not something users should employ unless
they have a good reason.

Post by Karl Dietz

Post by Max Barry
I know some users like to set
up multiple profiles, sometimes running a grabber with one config file,
sometimes with another, but surely that's fairly advanced behavior...
why must it be mandatory to support? It seems to encourage grabber devs
to simply shift the problem to users and always require them to remember
the right --config-file.

I think you're confusing the requirement for grabbers to *support* --config-file and the need for users to use this *optional* parameter. Just because a grabber 'supports' a parameter doesn't make it 'required'.

Right! And to my thinking, only a few advanced users actually do want to
maintain multiple config files. The majority just want to configure the
grabber once. However, the XMLTV wiki instructions tell users, "It is
recommended that you always use this parameter." Which is about as close
to "required" as can be!

What I was getting at with the above paragraph, though, is that it seems
to me that grabbers can do configuration in four ways, from simplest
(for grabber developers) to most complex:

1. Grabber does not permit user configuration at all.

2. Grabber allows configuration and always requires a --config-file
option, leaving it to users to remember what they choose and supply
every time thereafter.

3. Grabber allows configuration and uses a single default config file,
not allowing users to specify one.

4. Grabber allows configuration and accepts --config-file but does not
require it; in its absence, it uses a default.

And #3 is illegal according to XmltvCapabilities. As a grabber
developer, you have to go directly from #2 to #4. Which is kind of a big
deal, because #4 has several usability problems that #2 and #3 don't. A
few examples:

- Users accidentally winding up with multiple config files, leading to
confusion as to why their grabber is running differently sometimes than
others

- Users forgetting where they put their original config file when they
try to interact with the grabber

- If the grabber uses multiple config files (e.g. one for channels, one
for the grabber itself), these need to be kept in sync

- If the user replaces her grabber, there can be cron jobs or MythTV
scheduled jobs or whatever hanging around that call something like
"tv_grab_au --config-file ~/.xmltv/config.conf", invoking the new
grabber with the old config file, creating unpredictable results.

These problems led to my grabber dropping support for --config-file,
which was annoying for those advanced users who wanted to maintain
multiple config files, but made life much easier for everyone else.

So I was kind of complaining that #3 should be legal. But assuming that
is unlikely to change, my options are:

(A) Drop back to #2, requiring users to always specify --config-file.
This isn't really an option, since it's so user-unfriendly. It would
break everyone's existing installation.

(B) Support --config-file again.

(C) (Possibly?) Stop claiming "manualconfig" in response to
--capabilities. This seems compliant with XmltvCapabilities, but
inaccurate, since the grabber would indeed support (and require!) manual
configuration.

I was all prepared to do (B) until I saw the XMLTV wiki telling users to
always use --config-file. Then I had visions of the "HELP ME!" posts
rolling into my mailing list, with users following these directions and
accidentally winding up with multiple config files.

I am still thinking that's the best option, but with big scary warnings
for users who run "--configure --config-file <file>", saying something
like, "ARE YOU SURE YOU WANT TO DO THAT, YOU'RE GOING TO HAVE TO SUPPLY
THAT PARAMETER EVERY TIME YOU CALL ME FROM NOW ON, WHY DON'T YOU JUST
USE THE DEFAULT?" Which is not ideal because that advice will conflict
with the XMLTV wiki.

Max.

[h***@gmail.com]

I think you are over-thinking this.

Post by Max Barry
I was all prepared to do (B) until I saw the XMLTV wiki telling users to
always use --config-file.

No it doesn't say that. It is "recommended". This is so the user then knows for certain where their config file is located. Otherwise they are likely to have no idea (the grabber won't tell them!) and have to go searching for it (or creating one of your "HELP ME!" posts to ask where it is).

Post by Max Barry
Then I had visions of the "HELP ME!" posts
rolling into my mailing list, with users following these directions and
accidentally winding up with multiple config files.

In practice this is unlikely. I think you are underestimating the intelligence of your users. It's obvious to most if not all people that if, when they run the grabber, it says

"You need to configure the grabber by running it with --configure"

then they will know this is wrong and will think to check the parameters they specified (and/or use --help for clues).

It's trivial for you to add at the start of your 'config_stage' a simple message stating your grabber is about to create a new config file and reminding the user about the ---config-file option.

[Max Barry]

Post by h***@gmail.com

Post by Max Barry
I was all prepared to do (B) until I saw the XMLTV wiki telling users to
always use --config-file.

No it doesn't say that. It is "recommended".

Well yes, but it is "always" recommended. There are no circumstances in
which it is not recommended. The point is --config-file is presented not
as an optional parameter that some people might find useful, but as the
default everyone should use. Yet this can lead to highly sub-optimal
outcomes when the grabber doesn't require --config-file.

Post by h***@gmail.com
This is so the user then knows for certain where their config file is located.

It *creates* the requirement that the user know where their config file
is located. If their grabber supports a default, they don't need to care.

Post by h***@gmail.com
Otherwise they are likely to have no idea (the grabber won't tell them!) and have to go searching for it (or creating one of your "HELP ME!" posts to ask where it is).

Actually, the grabber may well tell them; mine mentions which config
file it's using in a few different places, including at the top of each run.

But why do users need to know where their config file is? I can't think
of a single occasion of a user asking me this. (Although since it's
always in a default location, maybe they're finding it by themselves.)

If there is a need for the user to manually poke around in the config
file, though, consider the difference between these two situations:

(A) User didn't supply --config-file and wants to find it. User runs
"--help" or checks the FAQ to find the default location.

(B) User did supply --config-file and wants to find it. The best help
they can be given is: "Try to remember where you put it when you setup
the grabber." It could be in any location under any name.

Post by h***@gmail.com

Post by Max Barry
Then I had visions of the "HELP ME!" posts
rolling into my mailing list, with users following these directions and
accidentally winding up with multiple config files.

In practice this is unlikely.

It was through actual experience that we dropped support for
--config-file, so this isn't all theoretical. Although admittedly the
main problem we had was people migrating from older grabbers and
accidentally running the new one under an old config file, rather than
creating multiple config-files for the new grabber, which is my concern now.

Post by h***@gmail.com
I think you are underestimating the intelligence of your users.

I don't think users should have to be smarter than necessary. Or have
excellent memories! If the grabber will remember where its config file
is, why make users do it? Years can pass between the grabber being
installed and the user deciding to poke at it.

Post by h***@gmail.com
It's obvious to most if not all people that if, when they run the grabber, it says
"You need to configure the grabber by running it with --configure"
then they will know this is wrong and will think to check the parameters they specified (and/or use --help for clues).

I think the situation you're describing here (please correct me if I'm
wrong!) is that a user supplied --config-file in the past but
subsequently forgets it and runs the grabber without it? If so, this is
indeed one possible outcome. (And a bad one, since as per the above,
that config file could be anywhere under any name. And the grabber is
telling them they haven't configured when in fact they have, they've
just forgotten where they put the config file.) Another possible
outcome, though, is that the default config file exists, having been
created by the user or some program in the past and since neglected, so
the grabber will run, but with unpredictable results. Which is really
awful and hard to debug.

Post by h***@gmail.com
It's trivial for you to add at the start of your 'config_stage' a simple message stating your grabber is about to create a new config file and reminding the user about the ---config-file option.

Yes, but I don't want users to have the XMLTV wiki telling them, "it is
recommended you always do this" and the grabber saying "don't do it
unless you really mean it!" Users will probably trust the grabber, but
it's not a great situation. Particularly since the wiki doesn't say
*why* users should always use --config-file, meaning you can't make an
informed decision.

I really can't see the sense in that wiki instruction, and unless I'm
still missing something, I think it should be removed. The correct
advice is that users should only use --config-file if it's mandatory for
the grabber or if they have a particular need for it.

Max.

[h***@gmail.com]

Post by Max Barry
These problems led to my grabber dropping support for --config-file,
which was annoying for those advanced users who wanted to maintain
multiple config files, but made life much easier for everyone else.

Actually I've just noticed that the Wiki *is* wrong but only in that it says --config-file should be implemented as part of 'manualconfig' capability. In fact --config-file is 'baseline' capability.

(see 'cap_options' in Options.pm and run-time procedure for ValidateGrabber.pm)


Note: this *does* make sense to me - a grabber can use a config-file but that doesn't mean it has to manually configured ( - it could be preset, or determined at installation time and then not user alterable).

Cheers,
Geoff

[h***@gmail.com]

Post by Max Barry
P.S. Somewhat relatedly, I'm curious as to why a grabber MUST support
--config-file if it supports --configure.

Consistency across grabbers. Avoids the user having to check whether a grabber supports --config-file; if it supports --configure then the user *knows* it supports --config-file

If a grabber requires a config file (and not all of them do) then it's reasonable to allow the user to change its file location.

Post by Max Barry
I know some users like to set
up multiple profiles, sometimes running a grabber with one config file,
sometimes with another, but surely that's fairly advanced behavior...
why must it be mandatory to support? It seems to encourage grabber devs
to simply shift the problem to users and always require them to remember
the right --config-file.

I think you're confusing the requirement for grabbers to *support* --config-file and the need for users to use this *optional* parameter. Just because a grabber 'supports' a parameter doesn't make it 'required'.

Regards,
Geoff