NAME

netcomics - retrieve comics from the Internet


SYNOPSIS

netcomics [-aAbBDGhiIlLoPsuvv] [-[c|C] ``comic ids''] [-p url] [-S,-T,-E date] [-R retries] [-n,-N days] [-d,-m dir] [-g [program]] [-nD] [-r rc_file] [-P] [-W,-w[=n]] [-nw] [-[n]wa] [-wt ``webpage title''] [-f,-wf ``date fmt''] [-t theme]


DESCRIPTION

The netcomics program will download today's comic strips from the Web, and place them in a spool directory where they can be retrieved for display. Because each website that carries comic strips chooses how old of strips to show, the comic strips downloaded will actually be from different dates. Most common will be comic strips that are 1 week, 2 weeks, and 3 weeks old. Also, each website is not updated at the same time, nor are any of them updated at a consistent time during the day. Therefore, when running netcomics as a cron job in the early morning, you may need to rerun it by hand a little later in the day to get the comics that it couldn't find. netcomics will automatically determine what comics need to be retried to be downloaded (max retries defaulted to 3).

netcomics also supports retrieving specific dates of comics. A ``Starting Date'' may be specified with -S. An ``Ending Date'' may be specified with -E. And a specific date may be specified with -T. All dates are given in the M-D-[YY]YY format. If a start date is given without an end date, -n may be used to specify the number of days of comics to retrieve starting at the specified start date. If an end date is given without a start date, -n will specify the number of days of comics to retrieve counting backwards from the end date. If a start date is specified with -S, but neither -n nor -E are used, then the end date of the range is assumed to be today's date. If the option given to -E, -S, or -T is just an integer, that integer is interpreted as the number of days prior to the current day (i.e. you can specify specific dates relatively).

Another way to specify the dates to download comics, is to use -N in conjunction with -n. The argument to -N is the number of days prior to the current day to retrieve comics. Note that this does not specify an actual date. It rather indicates the number of days ago a comic was made available, rather than the actual date of the comic. To see how far behind the date of each comic is from the date it actually gets released, use -l. So if today is Wednesday, and you specify -N 2, netcomics will download the comics that were made available on Monday. This is useful for running netcomics in a timezone that is way ahead of the timezones the comics' websites are located.

netcomics was created for the purpose of giving your weary mind a little relief from your hectic workday, so another script called show_comics has been provided to periodically popup the retrieved comic strips throughout the workday. Another script called display_comics has also been provided, but it is only an example script of what can be done for displaying comics on many users computers on a network.

Important: The further east your timezone is from the US, the later in the day you'll have to run netcomics. As a reference, I suggest those whose timezone is GMT to wait to run the script at 12:30pm if they want to get all the comics at a time that is likely to have most of the websites updated. Use -N if you want to run the script early in the morning and are having problems getting comics to download. Also, just because a comic failed to download doesn't mean that the module for that comic is broken--it may mean the website just hasn't been updated yet.

netcomics can also create an set of HTML files, in the directory you have the comics placed. If a number is given with -W or -w, it will be used to determine the number of comics to be placed in each html file. Each file is named ``comic#.html'', and an ``index.html'' file is created with a table of pointers into these files. You may specify the HTML theme using -t.

netcomics can also maintain archives of comics for you with the use of the -P feature. Using this will separate all comics of a single title into their own separate directories. Using the -P also effects the behaviour of the generation of HTML pages: for each comic, a different set of HTML pages is generated. It is a better idea to set the variable $seperate_comics to 1 in your netcomicsrc file so this option is not intermittently used.

Disclaimer: Do not put the comics up on the Internet! You should only use them for your own use. Also, do not redistribute the comics downloaded by netcomics in any other way unless you receive written authorization from each comic's publisher and/or author.


OPTIONS

-a
Always download even if the comic strip about to be downloaded exists in the spool directory. When netcomics starts up, it scans the download directory and reads in all the status information files for each comic strip in the directory (use -D to have netcomics clear the directory when it starts). By default, netcomics will skip downloading comics you specify to download if they are in the directory and they have a good status indicator. By specifying this option, netcomics will skip this step and clobber the previously downloaded comics.

However, if used with -c, the behavior is different. Netcomics by default will only download the comics you specify with -c. If -c isn't provided, netcomics will download everything. If both -a and -c are provided, it will process both the comics you specify (even if they were already successfully downloaded) and the ones that weren't successfully downloaded. Note that only the tries counter is ignored for the specified comics in this case. See -nR for changing the maximum number of retries.

-A
Consider today is the day specified with the -S, -T or -E options. Will recalculate dates decalage accordingly. This option only makes sense when used with the -S, -T or -E options.

-b
Get black & white comics if their is a choice between that and color.

-B
Don't get black & white comics if their is a choice between that and color. Use this option to override the setting in the rc file.

-c 'comic_ids'
Get the supplied comics (ids are separated by white spaces). This option may be repeated, and may not be used in conjunction with -C.

-C 'comic_ids'
Don't get the supplied comics (ids are separated by white spaces). This option may be repeated, and may not be used in conjunction with -c.

-d dir
Put comics into directory. Default is /var/spool/netcomics.

-D
Delete files in directory before retrieving.

-E [date | days]
Specify an ending date, or the date that is the specified number of days prior to today, with which to define the range of days of comics to retrieve. Must be used in combination with one of -S or -n. The date is of the form: M-D-Y.

-f date_fmt
Specify the date format used when naming files. Default is '%Y%m%d'.

-g [program]
Specify a program to use to perform HTTP requests. The program must write the retrieved content of the URL on its stdout. The macro %R is replaced with the URL of the referer, %P is replaced with the proxy URL (if supplied), and %U is replaced with the URL to retrieve. If %U is not found, the URL is appended (with a space). Macros are case insensitive.

Note that you don't have to specify the program on the command line if you want to use the default command which is:

   'wget -q -O - --header="Referer: %R"'.

You'll probably just want to set $external_cmd in your rc file. Note that when no %P is in the string and a proxy is specified, the command will be prepended with ``env http_proxy=<proxy> ''

-G
Puts all rli information into one .rli file, named .netcomics.rli by default. Like -P, it is a good idea to set $single_rli_file in your netcomicsrc file instead of using this from the command line.

-h
Show usage. Comics will not be downloaded.

-i
Don't create an index for the webpages.

-I
Create an index for the webpages. Specify this to override the setting in the rc file if need be.

-l
List supported comics & their identifiers, sorted by name alphabetically. Comics will not be downloaded.

-lt
List the names of the HTML themes. Comics will not be downloaded.

-L
Sort comics in webpages by date and when using -l, by days behind.

-m dir
Add dir to the locations of comic modules. Default is /usr/share/netcomics. This option may be repeated to add multiple directories.

-M dir
Use only the list of locations of comic modules specified on the command line.

-n days
Retrieve this number of days of comics, going backwards. Default is 1. This option may be used in conjunction with -N.

-N days
Start retrieving comics this many days before the currently available date. If you use -l to show the comic id's, the 3rd column indicates the number of days behind a comic is available. By default, if -E or -S are not specified, then netcomics will retrieve each comic, understanding that the latest available is that many days ago (according to the number shown in the 3rd column). Use this option to push the number of days ago back even further. Default is 0. This option may be used in conjunction with -n, but not with -S or -E.

-o
Write the webpage on standard out. Cannot specify the number of comics per page for -W or -w if you use this option.

-p url
Specify a URL to use as a proxy. Both HTTP and FTP are supported. If you use wget, specify http_proxy = URL or ftp_proxy = URL in your .wgetrc file.

-P
Turns on comic separation mode, which puts comics into their own subdirectories. This is useful for maintaining an archive of comics. It is a better idea to set the $separate_comics variable to 1 in the netcomicsrc file so that this feature is not intermittently used.

-Q
Supresses the final error message that prints out the command to rerun to attempt to try to retreive the comics that failed to download.

-nQ
Disable suppression of final error message (override rc file setting).

-q
Show what comics would be downloaded. Don't actually do anything.

-r rc filename
Specify an alternate name for the user-specific rc file

-R retries
Specify the maximum number of retries for downloading comics not specified to be downloaded, inbetween invocations of netcomics. If you don't have the download directory cleared out, netcomics will pick up information on comics it's tried to download previously. Specify 1 to this option if you don't want netcomics to retry to download comics you don't specify for it to download. Specify 0 for an infinite number of retries.

-nR
Specify there is no maximum number of retries. This will cause netcomics to keep trying to download comics that haven't been successfully downloaded no matter how many unsuccessful attempts there were. This is the same thing as -R 0.

-s
Skip bad comics when creating the web page. The default behavior will potentially cause the web page to be loaded into a browser more slowly, so using this option will keep broken embedded image links from being included, possibly speeding up loading of the webpage.

-S [date | days]
Specify a starting date, or the date that is the specified number of days prior to today, with which to define the range of days of comics to retrieve. May be used in combination with of -E or -n. The date is of the form: M-D-Y.

-t theme
Specify the name of the HTML template theme to use. Theme files are located in /usr/lib/perl5/site_perl/5.8.0/Netcomics/HTML/Themes. Use -lt to list the installed themes.

-T [date | days]
Specify a specific date, or the date that is the specified number of days prior to today, of comics to retrieve. This option my be repeated. The date is of the form: M-D-Y.

-u
Don't download comics, but print URLs on stdout, or if creating a webpage, make the images implemented using the URLs. Note that this option can be used with -a (always download). This option doesn't keep everything from being downloaded--just the final URL. -a simply turns off netcomics' tendency to needlessly redownload comics already ``downloaded'' (whether it be fully downloaded or only to the point of having a URL to the comic strip).

Another important note is that some websites try to prevent downloading of comics from programs like this by requiring the ``referer'' of the HTTP Get operation to be of a particular website. For those websites that impose this policy, using this option will result in comics that won't download when browsing the webpage netcomics produces because the referer your browser will specify when doing the HTTP Get operation for those comics won't be the right one.

-v
Be a little verbose.

-vv
Be extra verbose

-w[=n]
Create an html file, index.html, for the comics downloaded. Optionally, n specifies the number of comics to have in each page, where subsequent html files are named comic#.html.

-nw
Do not create a webpage. Use this option to override the rc file setting, make_webpage, if need be. When no webpage is created, filenames of the selected comics will be printed. If the -u option is used, URLs will be printed instead. Use -v also to include the comic tiles before the filenames or URLs.

-wa
In the created webpages, use absolute paths for embedded images.

-nwa
In the created webpages, use relative paths for embedded images. Use this option to override the rc file setting, $webpage_absolute_paths.

-wt title
Specify a title for the web page rather than the default (``Today's Comics From the Web on <DATE>''). This is useful for when you download specific comics, and want the title of the web page to reflect the actual contents.

-wf date_fmt
Specify the date format used to replace occurrences of <DATE> when creating webpages. Default is '%b %d, %Y'. See the strftime(3) manpage for a list of available conversion specifiers on your system.

-W[=n]
Recreate the html file, index.html, from the comics that are in the directory, as well as any new comics downloaded. Optionally, n specifies the number of comics to have in each page, where subsequent html files are named comic#.html.


EXAMPLES

  1. Run as a cronjob.
  2. Run as a cron job at 7:30am, Monday through Friday, removing the previous day's comics beforehand, and creating a web page. And for Monday, also retrieve Saturday & Sunday's comics.
       30 07  *  *  2-5  /usr/bin/netcomics -D -w
       30 07  *  *  1    /usr/bin/netcomics -n 3 -D -w

  3. Run as a cronjob, not so many on Monday.
  4. Same as before, except, for Monday, get Saturday's & Sunday's comics, and for Tuesday, get Monday & Tuesday's. This is so there isn't such an overload of comics on Monday.
       30 07  *  *  1    /usr/bin/netcomics -N 1 -n 2 -D -w
       30 07  *  *  2    /usr/bin/netcomics -n 2 -D -w
       30 07  *  *  3-5  /usr/bin/netcomics -D -w

  5. Run as a cronjob, email yourself the URLs.
  6.    30 07  *  *  *    /usr/bin/netcomics -uv | mailx -s "today's comics" $LOGNAME
  7. Select comics, separate directory, different date format in webpage.
  8. Grab Dilbert & Foxtrot comics from the past 30 days, place them in /tmp, and create a web page with a specific title (<DATE> gets replaced with the name of the month).
       netcomics -c "dilbert ft" -n 30 -d /tmp -w -wt 'Dilbert & \
       Foxtrot Comics From the Month of <DATE FORMAT="%b">'

  9. Date range, start to finish, plus an extra date.
  10. Specify the date range of comics to retrieve to be from Feb 3, 1999 to Feb 6, 1999, and also get comics on March 3, 1998.
       netcomics -S 2-3-99 -E 2-6-99 -T 3-3-98

  11. Date range, specify end & number of date, exclude some comics.
  12. Specify the date range of comics to retrieve to be from Jan 6, 1999 and the 5 days before it. Get all the comics except Jerkcity and Doodie
       netcomics -E 1-6-99 -n 6 -C jc -C doodie

  13. Specify number of days of comics, a number of days ago.
  14. Specify the date range of comics to retrieve to be all those that came available three, four, and five days ago.
       netcomics -N 3 -n 3

  15. Specify number of days of comics, a number of days ago.
  16. Specify the date range of comics to retrieve to be all those that are dated three, four, and five days ago. This example is given to show the difference between -E & -S (when given a number) and -N. All comics downloaded will have the same 3 dates, while in the previous example, the comics will have varying dates that are determined by the 3rd column in the output of -l. Note that since many comics aren't available until 2 weeks have past, many of them will not download with this example.
       netcomics -E 3 -n 3
                or
       netcomics -E 3 -S 5

    See -A for a more elegant way of downloading comics released in the past.

  17. Use wget.
  18. Use wget instead of libwww-perl (makes it so you don't have to install libwww-perl).
       netcomics -g

  19. Print webpage with URLs to stdout.
  20. Do not actually download the comics. Output the webpage (which will point to the comics on the websites they actually are released from) to stdout.
       netcomics -uwo

  21. Pretend today's date is a different one.
  22. Download the Calvin & Hobbes, Alice and Dragon Tails as if the day the download was done was May 1st 2000.
       netcomics -T 5-1-00 -A -c "ch alice dt"

    This will download Dragon Tails for May 1st 2000 since the strip is available the same day, but will download the May 1st 1989 Calvin & Hobbes' strip since it is 4018 days back, as well as the April 17th 2000 strip for Alice since it is 14 days back.

    This behaviour also applies for the -E and -S options.

  23. Forcibly try to download those that failed.
  24. Have netcomics retry downloading comics it didn't successfully download, regardless of the number of times they've been retried (also remaking the webpage).
       netcomics -c "" -R 0 -W

  25. Forcibly redownload everything.
  26. Have netcomics ignore all previously downloaded (and not-yet downloaded) comics and redownload everything. Note that this was netcomics' default behaviour prior to version 0.13. Also note this has the affect of resetting all comics' retry counter to 1.
       netcomics -a

  27. List the supported comics.
  28. List all the supported comics, including their IDs that are used with the -c option, and how old they are before being released on the website from which they are downloaded.
       netcomics -l

  29. Override rc file options.
  30. With a ~/.netcomicsrc file having $delete_files = 1 and $remake_webpage = 1 or $make_webpage = 1, tell netcomics to not delete the previously downloaded comics and to not create or recreate the webpage.
       netcomics -nD -nw

  31. Create your own archives of comics.
  32. Tells netcomics to download Kevin & Kell and User Friendly for the last fifteen days, place them in separate directories, and then make a webpage with 10 entries on each page.
            netcomics -n 15 -W=10 -P -c "kk uf"

  33. Recreate the webpages with image URLs as absolute paths
  34.         netcomics -c '' -W -R 1 -wa -vv


FILES

/usr/lib/perl5/site_perl/5.8.0/Netcomics/Comic
Directory containing the comic modules.

/usr/lib/perl5/site_perl/5.8.0/Netcomics/HTML/Themes
Directory containing the HTML template theme files of which one is used to create the web page. Each file contains a different set of templates.

/usr/lib/perl5/site_perl/5.8.0/Netcomics/etc/netcomicsrc
System rc file. This can be copied for each user's own .netcomicsrc file.

~/.netcomicsrc
User rc file used to change the default settings.

/var/spool/netcomics
Default directory where comics and web pages are placed.

/usr/bin/display_comics
Example script that should be modified to be used to display the downloaded comics.


BUGS

See the TODO file in the source distribution.


SEE ALSO

show_comics(1) comicpage(1) strftime(3) wget(1) env(1)


AUTHOR

Netcomics Maintainers <netcomics-maintainers@lists.sf.net>