Content-type: text/html Manpage of NEWSLEECH

NEWSLEECH

Section: User Manuals (1)
Updated: July 2000
Index Return to Main Contents

NAME

newsleech - non-interactive NNTP news retriever

SYNOPSIS

DESCRIPTION

Newsleech is an NNTP client that automatically retrieves groups of articles or lists of articles and sends them to standard output, files, the standard input of some unix program. It can visit multiple servers with multiple newsgroups and get multiple groups of articles, each with it's own destination (file, stdout, pipe or default).

It gets it's information about what to do from the commandline and/or from a simple script (see below in section SCRIPTING ). The order of commandline options is very important. The options before the first server or group specified change the default retrieval mode: --article, --body, --header and --list, and the default action: --stdout, --file FILENAME, --pipeto COMMAND. These defaults apply to each group of articles unless --range RANGE is directly prepended by at least one of these options. (confusing? please take a look at the EXAMPLES below)

After you specified one range of articles for a group, you can specify another and another..... or you can move to a different group or server. Note that if, for example, you want to visit the same group on three different servers, you have to specify that group three times (one time for each server).

OPTIONS

-v, --verbose LEVEL

This option sets the verbosity level.

-v1 allows you to see the NNTP conversation between newsleech and the NNTP server.

-v2 shows you information about the TCP connection, like telnet does.

-v4 displays the process of commandline/script parsing in scriptform. This is very educational if you want to write your own scripts. This has to be the very first option specified, ortherwise verbose parsing is disabled.

-v8 tells you exactly wich files are created and what commands are executed.

Verbositylevels are additive. (so -v5 means: show NNTP conversation and enable verbose parsing) All messages are sent to standard error so standard output will not be clobbered.

--dummy

This option ensures that the program exits after scanning the commandline (or parsing the script). It can be useful in combination with -v4.

-s, --server SERVER

Use this option to specify a news server.

-p, --port PORTNUM

Use this option directly after you specified the server to use another port. The standard NNTP port is 119.

-i, --interval INTERVAL

Some, mostly free, servers have a limit on the number of connections. When the maximum number of connections is reached they give an error like '502 Too much active connections' or so. This option lets the program reconnect every INTERVAL seconds. It will do so indefinitely. Newsleech does not pay attention to the text response of the server; on '502 You have no permission to talk. Goodbye.' it will also retry indefinitely. The default is 0: do not retry. Use this option with caution.

--aretry RETRIES

Sometimes, a newsserver lies about the availability of an article. It tells us 'no such article' while the article is available. With this option you can ask newsleech to retry asking for the article RETRIES times. The default is 0, i.e. do not retry.

--username USERNAME, --password PASSWORD

These options let you specify your username and password for a server that requires authentication. It is generally a bad idea to do this on a multi-user system because everybody who types 'ps ax' can see it. A better option is to use the scipting facility (be sure to deny readaccess to your script from group and others).

-g, --group GROUP

With this option you can specify a newsgroup.

-r, --range RANGE

If the retrieval mode is --article, --body, or --header then RANGE is a comma seperated list of message numbers and (what normal people call) ranges of message numbers or single message numbers. For example 134,136-140,133. There should be no typographical spaces in RANGE or else this option will be misinterpreted.

If the retrieval mode is --list, then RANGE must be either a single message number or a single message number followed by a dash ('-') or a single message number followed by a dash followed by another message number or the word 'all'.

-a, --article

Set retrieval mode to --article, this means that of all messages in the following --range RANGE the whole message will be fetched. This is the default.

-b, --body

Set retrieval mode to --body, this means that of all messages in the following --range RANGE only the body will be fetched. If you specify this option before any servers and groups; the default retrieval mode will be set to --body.

-e, --header

Set retrieval mode to --header, this means that of all messages in the following --range RANGE only the header will be fetched. If you specify this option before any servers and groups; the default retrieval mode will be set to --header.

-l, --list HEADER

Set retrieval mode to --list, this means that of all message in the following --range RANGE all headerlines (if any) starting with HEADER will be fetched. If you specify this option before any servers and groups; the default retrieval mode will be set to --list HEADER. This option will only work if the news server understands the XHDR command.

--stdout

Dump the articles in the following --range RANGE to standard output. This is the default.

-f, --filename FILE

Send the articles in the following --range RANGE to FILE. If you specify this option before any servers and groups; the default filename will be set to FILE.

-t, --pipeto COMMAND

Pipe the articles in the following --range RANGE to the standard input of COMMAND. If you specify this option before any servers and groups; this is made the default.

--script FILENAME

Read commands (actually long options arguments) from the file FILENAME and interpret them as if they appeared on the commandline. Lone files on the commandline are also interpreted as scripts, I have provided the --script option for readablility. The script option can not be used in scripts.

SCRIPTING

If there is no server (explicit of implicit) on the commandline, newsleech will expect a script on standard input. This doesn't mean that you can use the program interactively; processing starts after the whole commandline/script has been parsed.

You can also specify a script almost anywhere (not between an option and it's argument) on the commandline. The commands in the script will be interpreted as if the appeared on the commandline at the place the script is specified.

The commands are the same as the long-options discussed above. You can't use version , help and verbose however. You can use end to mark the end of the script. It is not neccesary to use it. Blank lines and comments are allowed. Comments must have a # at the beginning of the line. It is not possible to put comments on the same line as commands. If there is whitespace in option arguments you must enclose the arguments in either single or double quotes.

The scripting language is not documented here because there are numerous examples available. To see an example look at the output of -v4 when you give some sensible commandline options. There is nothing what you can do with a script that you can't do from the commandline, except for the limited lenght of the commandline. See the section EXAMPLES for some examples.

EXAMPLES

Some examples:

$ newsleech -g alt.os.linux -l Subject -r all

will dump a list of subject lines of all the messages in alt.os.linux on the newsserver pointed to by the environmentvariable NNTPSERVER.

Suppose that alt.binaries.whatever contains a one-part uuencoded binary posting with message number 35423, then

$ newsleech -g alt.binaries.whatever -f test.uu \
--body -r 35423

will put the body (the article without the header) in the file test.uu, or better

$ newsleech -g alt.binaries.whatever --pipeto=uudecode \
--body -r 35423

will pipe the body to uudecode(1) which will decode it.

Now a multipart posting, part 1 is at 1923, part 2 is at 1921, part 3,4 and 5 are at 1924,1925 and 1926. It is in uucode format. Suppose you also want a listing of subjects in alt.os.linux from an alternate server in file alt.os.linux.Subject, then

$ newsleech -v4 -g alt.binaries.whatever -t uudecode \
               --body -r 1923,1921,1924-1926 \

               -s alternate.news.server -g alt.os.linux \

              -l Subject -f alt.os.linux.Subject -r all

will do exactly what you want. Note that I send only the bodies of the articles to uudecode(1), if I had not done that uudecode(1) would have been confused. This command will also produce a script on standard error. It can serve as an example script. Traditional uudecode(1) doesn't cope very well with filenames with spaces, uudeview(1) is a good alternative.

Last example: suppose you must authenticate yourself to news.bla.com but you don't want to put your username/password on the commandline. What you can do is the following: Make a file bla.txt containing

server news.bla.com
username myusername
password therightpassword

(make sure the permissions on bla.txt aren't to liberal) and issue the command

$ newsleech --script bla.txt -g comp.os.netbsd .....

and you have access to your server without your username/password on the commandline. Note that you could have left the servername on the commandline instead of putting it in the script.

ENVIRONMENT

NNTPSERVER: If no server is given, the contents of this environmentvariable will be used as server. If you want to specify the portnumber, aretry, interval or username and password you must specify the server on the commandline.

DIAGNOSTICS

The error messages of this program are meant to be descriptive. If, however, this program does not seem to do what you asked, please turn on full verbosity ( -v15 ) and examine the output. You can, in addition, specify the --dummy argument to let newsleech only parse the commandline/script. Remember that -v15 must be the very first option on the commandline if you want to witness the parsing.

Every discrepancy between the documentation and the actual behaviour of the program is a bug in either the documentation or the program and should be treated as such.

AVAILABILITY

The latest version of newsleech should be available at
http://www.phys.uu.nl/~snel/newsleech/.

BUGS

File- and errorhandling could be better: Files will graciously be overwritten without notification and this program will probably hang forever on TCP errors and unanticipated replies from the NNTP server. A progress indicator would be nice.

Retrieval mode --list does not work with news servers that don't understand the XHDR command. Also some servers don't understand XHDR like I do and they give an error.

VERSION

The version of newsleech documented here is 0.1.0.

AUTHOR

Rik Snel <snel@phys.uu.nl>

THANKS

I wish to thank the authors of fetchmail(1) and wget(1) for GPL'ing their code. I have leared quite a lot from it and I took some ideas and scraps of code for use in newsleech.

This document was created by man2html, using the manual pages.
Time: 19:32:54 GMT, September 08, 2000