Discussion:
[Xmltv-devel] Something wrong, very high CPU usage
Dreamcat4
2015-03-23 15:43:01 UTC
Permalink
Hello!

I'm running xmltv from tvheadend, inside of a docker container.
However unfortunately I am suffering from high CPU usage. In my logs,
when the tvheadeend server invokes perl process tv_grab_uk_atlas, it
has some weird log messages:


https://gist.github.com/dreamcat4/58fd411e63ee0165a55c

Then for a long time it is making very high CPU usage with groups like this:

2015-03-23 13:44:01.914 [ ERROR] spawn: [Mon Mar 23 13:44:01 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.018 [ INFO] epgdb: saved
2015-03-23 13:44:02.018 [ INFO] epgdb: brands 0
2015-03-23 13:44:02.019 [ INFO] epgdb: seasons 0
2015-03-23 13:44:02.019 [ INFO] epgdb: episodes 861
2015-03-23 13:44:02.019 [ INFO] epgdb: broadcasts 861
2015-03-23 13:44:02.022 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.131 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.242 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.349 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.457 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.563 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.670 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.778 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.885 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.992 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:03.021 [ INFO] epgdb: saved
2015-03-23 13:44:03.021 [ INFO] epgdb: brands 0
2015-03-23 13:44:03.022 [ INFO] epgdb: seasons 0
2015-03-23 13:44:03.022 [ INFO] epgdb: episodes 861
2015-03-23 13:44:03.022 [ INFO] epgdb: broadcasts 861
2015-03-23 13:44:03.103 [ ERROR] spawn: [Mon Mar 23 13:44:03 2015]
Filtering duplicates: 99%
2015-03-23 13:44:03.212 [ ERROR] spawn: [Mon Mar 23 13:44:03 2015]
Filtering duplicates: 99%
2015-03-23 13:44:03.321 [ ERROR] spawn: [Mon Mar 23 13:44:03 2015]
Filtering duplicates: 99%
2015-03-23 13:44:03.428 [ ERROR] spawn: [Mon Mar 23 13:44:03 2015]
Filtering duplicates: 99%
2015-03-23 13:44:03.536 [ ERROR] spawn: [Mon Mar 23 13:44:03 2015]
Filtering duplicates: 99%
2015-03-23 13:44:03.625 [ ERROR] spawn: display-name element: not
writing undefined value for display-name at /usr/share/perl5/XMLTV.pm
line 1830.
2015-03-23 13:44:03.625 [ ERROR] spawn: bad data inside channel
element, not writing
2015-03-23 13:44:04.018 [ INFO] epgdb: saved
2015-03-23 13:44:04.018 [ INFO] epgdb: brands 0
2015-03-23 13:44:04.019 [ INFO] epgdb: seasons 0
2015-03-23 13:44:04.019 [ INFO] epgdb: episodes 861
2015-03-23 13:44:04.019 [ INFO] epgdb: broadcasts 861
2015-03-23 13:44:05.018 [ INFO] epgdb: saved
2015-03-23 13:44:05.018 [ INFO] epgdb: brands 0
2015-03-23 13:44:05.018 [ INFO] epgdb: seasons 0
2015-03-23 13:44:05.018 [ INFO] epgdb: episodes 861
2015-03-23 13:44:05.019 [ INFO] epgdb: broadcasts 861
Dreamcat4
2015-03-23 15:46:20 UTC
Permalink
Just to say:

I am glad to be making enough progress to know that CPU usage dropping
back to normal once the xmltv grab has finished. However the error is
not found on google. So I post here.

Perhaps the next thing I shall try is a clean new atlas config. Or
else disable XMLTV entirely. I'm not sure yet which solution is best.

But in any case I may try to enable more logging in tvheadend. Is
there a log file for the XMLTV perl script too ?
Post by Dreamcat4
Hello!
I'm running xmltv from tvheadend, inside of a docker container.
However unfortunately I am suffering from high CPU usage. In my logs,
when the tvheadeend server invokes perl process tv_grab_uk_atlas, it
https://gist.github.com/dreamcat4/58fd411e63ee0165a55c
2015-03-23 13:44:01.914 [ ERROR] spawn: [Mon Mar 23 13:44:01 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.018 [ INFO] epgdb: saved
2015-03-23 13:44:02.018 [ INFO] epgdb: brands 0
2015-03-23 13:44:02.019 [ INFO] epgdb: seasons 0
2015-03-23 13:44:02.019 [ INFO] epgdb: episodes 861
2015-03-23 13:44:02.019 [ INFO] epgdb: broadcasts 861
2015-03-23 13:44:02.022 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.131 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.242 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.349 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.457 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.563 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.670 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.778 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.885 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:02.992 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
2015-03-23 13:44:03.021 [ INFO] epgdb: saved
2015-03-23 13:44:03.021 [ INFO] epgdb: brands 0
2015-03-23 13:44:03.022 [ INFO] epgdb: seasons 0
2015-03-23 13:44:03.022 [ INFO] epgdb: episodes 861
2015-03-23 13:44:03.022 [ INFO] epgdb: broadcasts 861
2015-03-23 13:44:03.103 [ ERROR] spawn: [Mon Mar 23 13:44:03 2015]
Filtering duplicates: 99%
2015-03-23 13:44:03.212 [ ERROR] spawn: [Mon Mar 23 13:44:03 2015]
Filtering duplicates: 99%
2015-03-23 13:44:03.321 [ ERROR] spawn: [Mon Mar 23 13:44:03 2015]
Filtering duplicates: 99%
2015-03-23 13:44:03.428 [ ERROR] spawn: [Mon Mar 23 13:44:03 2015]
Filtering duplicates: 99%
2015-03-23 13:44:03.536 [ ERROR] spawn: [Mon Mar 23 13:44:03 2015]
Filtering duplicates: 99%
2015-03-23 13:44:03.625 [ ERROR] spawn: display-name element: not
writing undefined value for display-name at /usr/share/perl5/XMLTV.pm
line 1830.
2015-03-23 13:44:03.625 [ ERROR] spawn: bad data inside channel
element, not writing
2015-03-23 13:44:04.018 [ INFO] epgdb: saved
2015-03-23 13:44:04.018 [ INFO] epgdb: brands 0
2015-03-23 13:44:04.019 [ INFO] epgdb: seasons 0
2015-03-23 13:44:04.019 [ INFO] epgdb: episodes 861
2015-03-23 13:44:04.019 [ INFO] epgdb: broadcasts 861
2015-03-23 13:44:05.018 [ INFO] epgdb: saved
2015-03-23 13:44:05.018 [ INFO] epgdb: brands 0
2015-03-23 13:44:05.018 [ INFO] epgdb: seasons 0
2015-03-23 13:44:05.018 [ INFO] epgdb: episodes 861
2015-03-23 13:44:05.019 [ INFO] epgdb: broadcasts 861
h***@gmail.com
2015-03-23 19:24:31 UTC
Permalink
Post by Dreamcat4
I am glad to be making enough progress to know that CPU usage dropping
back to normal once the xmltv grab has finished. However the error is
not found on google. So I post here.
Perhaps the next thing I shall try is a clean new atlas config. Or
else disable XMLTV entirely. I'm not sure yet which solution is best.
But in any case I may try to enable more logging in tvheadend. Is
there a log file for the XMLTV perl script too ?
Hi,
You don't say whether this is a new install or something which was previously working, but in event I can't help you here I'm afraid as I don't know TVH. I suggest you ask on the TVHeadend forum and see if they can point you at where to look in TVH for the XMLTV config file/cache files/logs. Once you've found those you can try running tv_grab_uk_atlas manually and see if your error still exists.
Post by Dreamcat4
display-name element: not writing undefined value for display-name
suggests that your setup in TVH is wrong (perhaps you are asking for a channel which does not exist in tv_grab_uk_atlas ?)
Dreamcat4
2015-03-23 20:18:02 UTC
Permalink
Post by h***@gmail.com
Post by Dreamcat4
I am glad to be making enough progress to know that CPU usage dropping
back to normal once the xmltv grab has finished. However the error is
not found on google. So I post here.
Perhaps the next thing I shall try is a clean new atlas config. Or
else disable XMLTV entirely. I'm not sure yet which solution is best.
But in any case I may try to enable more logging in tvheadend. Is
there a log file for the XMLTV perl script too ?
Hi,
You don't say whether this is a new install or something which was previously working,
Well because its both of those things. Previous config files taken
from a working install (on FreeBSD). Then those files were copied into
a seeding folder for docker to create a clean instance of tvheadend
server.

This is good news because I can begin to easily debug which it is (the
config files or the docker container). Simply by trying a clean config
using `--configure` flag to setup from scratch. Other things I can try
too.

Like you say later on in your message - run outside tvheadend (from
command line) and compare. etc. Many things to try out.
Post by h***@gmail.com
but in event I can't help you here I'm afraid as I don't know TVH. I suggest you ask on the TVHeadend forum and see if they can
Yes understood. I'm just going to use this thread to document my
findings so anybody else who ever gets the same error message can find
out about it later on. Tvheadend people don't know either because this
message has not ever been seen before. And none of the main tvheadend
people are using the docker environment (yet).
Post by h***@gmail.com
point you at where to look in TVH for the XMLTV config file/cache files/logs. Once you've found those you can try running tv_grab_uk_atlas manually and see if your error still exists.
Post by Dreamcat4
display-name element: not writing undefined value for display-name
suggests that your setup in TVH is wrong (perhaps you are asking for a channel which does not exist in tv_grab_uk_atlas ?)
The channels to grab are listed in a config file in ~/.xmltv/ folder.
My previous FreeBSD config. I am pretty sure the tvheadend server
process it not involved with setting the .xmltv config files… AFIAKT
it just runs 'perl tv_grab_<whatever>' and accepts the standard
output. However I can definitely enable extra logging in tvheadend.
Especially if it's some kind of a process spawning issue.

I'm sure a few more investigations will narrow it down some!
Good day.
Post by h***@gmail.com
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
xmltv-devel mailing list
https://lists.sourceforge.net/lists/listinfo/xmltv-devel
h***@gmail.com
2015-03-24 07:43:03 UTC
Permalink
Post by Dreamcat4
Post by h***@gmail.com
but in event I can't help you here I'm afraid as I don't know TVH. I suggest you ask on the TVHeadend forum and see if they can
Yes understood. I'm just going to use this thread to document my
findings so anybody else who ever gets the same error message can find
out about it later on. Tvheadend people don't know either because this
message has not ever been seen before.
Just for info:

The things you noted such as

2015-03-23 13:44:02.022 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%

are not in fact an 'error' but are progress messages output by tv_grab_uk_atlas when the --quiet option is not being used. I assume TVH is calling them an 'error' because *it* doesn't know what they mean. (It should really call unknown messages "INFO" rather than "ERROR", IMHO)


I suggest you run tv_grab_uk_atlas manually (using your existing TVH config file) and see what comes out.

tv_grab_uk_atlas --config-file {config file path+name} --days 1 --offset 1 --output test_atlas.xml
tv_validate_file test_atlas.xml


(p.s. the current version of tv_grab_uk_atlas is v1.28)

(p.p.s. I'm just wondering if the progress message output could be the cause of your CPU grinding - check that TVH is calling tv_grab_uk_atlas with the "--quiet" option)
Dreamcat4
2015-03-24 09:59:08 UTC
Permalink
Post by h***@gmail.com
Post by Dreamcat4
Post by h***@gmail.com
but in event I can't help you here I'm afraid as I don't know TVH. I suggest you ask on the TVHeadend forum and see if they can
Yes understood. I'm just going to use this thread to document my
findings so anybody else who ever gets the same error message can find
out about it later on. Tvheadend people don't know either because this
message has not ever been seen before.
The things you noted such as
2015-03-23 13:44:02.022 [ ERROR] spawn: [Mon Mar 23 13:44:02 2015]
Filtering duplicates: 99%
are not in fact an 'error' but are progress messages output by tv_grab_uk_atlas when the --quiet option is not being used. I assume TVH is calling them an 'error' because *it* doesn't know what they mean. (It should really call unknown messages "INFO" rather than "ERROR", IMHO)
I suggest you run tv_grab_uk_atlas manually (using your existing TVH config file) and see what comes out.
tv_grab_uk_atlas --config-file {config file path+name} --days 1 --offset 1 --output test_atlas.xml
tv_validate_file test_atlas.xml
(p.s. the current version of tv_grab_uk_atlas is v1.28)
Thanks for these help ^^.
Post by h***@gmail.com
(p.p.s. I'm just wondering if the progress message output could be the cause of your CPU grinding - check that TVH is calling tv_grab_uk_atlas with the "--quiet" option)
Right. Well some very good news now:

After I turned off xmltv completely, it looked back to normal. But the
next day there was very high high CPU usage again. So that means it
was just a co-incidence the first time (while the grabber was running
and lots of so-called 'error messages). And xmltv is not the cause of
that after all. So actually it's tvheadend. Sorry for any confusion.

I can return to these log messages later on, if there proves to be
still any other remaining problem with xmltv. And hope to explain to
tvheadend project about the stderr channel not being any error (just
an information update). To get clearer log messages which don't show
up as errors. Sorry about that!
Post by h***@gmail.com
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
xmltv-devel mailing list
https://lists.sourceforge.net/lists/listinfo/xmltv-devel
Continue reading on narkive:
Loading...