Newsleech

WHAT IS IT?

This program can retrieve news messages by means of a simple and powerful 
command line interface. It can, in one run, visit several servers with 
several newsgroups with several sets of articles. It will concatenate this sets of articles and send the result to stdout, a file or a pipe. The program 
can recieve it's commands also from a script. In addition, newsleech can also 
retrieve list of header lines.

WHAT IS IT NOT?

The program, however, will never interpret any of the messages it receives.
That job is up to uudecode, mmdecode and 'intelligent' (perl-)scripts.

DO I NEED IT?

It depends. If you don't like the lack of control you have while using other
non-interactive retrieval programs and if you are prepared to write your 
own scripts (or ask your local guru's to do so) then this program is something
for you. Here one idea for a script:

-parthunt; such a script will visit a list of newsservers in search for a
	part of a binary posting, it will use newsleech to retrieve lists of 
	articles and the missing part (if it is located)

If anyone writes a nice script and contributes it to the project, then I will
put it on the project homepage.

Also, if you ever wanted to ask your computer:

'hey, get the header of message 19234 from bla.bla on server 
news.bla.net and put it in msg19234.txt',

or something similar. Then this program may also be something for you.

Note that this program is written for efficient retrieval of multiple postings
and not as a 'helper program' for a newsreader. (nothing prevents you from
using it this way however)

I wrote it because I have shell-access to a really nice, big machine with a
very nice connection to the internet and I wanted to selectively download
relevant binary postings. I could only find an interactive one (usenet binary 
harvester, I believe) and a non selective download tool (sucknews). Also, 
none of the other programs could give me a list of subjects which could be
analyzed by an external program. This program does exactly what I want and a
little bit more.

PORTABILITY

The program should be quote portable; it includes it's own getopt.c,
and vsnprintf() and gettext are not compulsory. Please comment one or two 
defines in config.h out if you don't have gettext and/or vsnprintf().

On a GNU/Linux system the compilation should give no errors. On the one other
platform I have tested (Digital Unix 64bit on a Compaq ES40). There are
essentially two warnings. One about a redeclaration of va_args in a header
file. And the other 'type mismatch in implicit declaration if builtin function
alloca'. If you are on Sun Solaris 5.6 (or something similar), try
Makefile.sol56.

PATCHES/FEEDBACK

You are welcome to submit patches, comments and questions and such to me. 
My email address is snel@phys.uu.nl . Please follow my codingstyle. If I 
accept you patch, and you didn't protest, you will be listed in the CREDITS 
file and eternal glory will be your part. ;-)  

The main principles of the program are:
-newsleech does not interpret bodies of messages in any way
-the command line user interface and the script facility must be
	flexible and consistent.

There is no development mailinglist at the moment. The homepage of the 
project is:
	http://www.phys.uu.nl/~snel/newsleech,
new releases (if they exist) can be found here.

SO HOW DO I COMPILE IT THEN?

Just do: make. You will get one executable. It's recommended to put the 
executable in /usr/local/bin and the manpage in /usr/man/man1. Edit config.h
if there are problems with vsnprinf and or gettext.

Note that you need GNU flex. (which can be downloaded from your local mirror 
of ftp.gnu.org/pub/flex)

WHERE CAN I FIND DOCUMENTATION

User documentation is in the manpage. Here I will write a short piece in the
inner workings of the program.

INNER WORKINGS

The program starts by parsing the commandline or the script. During this 
phase the program fills a set of structures. 

First the 'rootnode', this structure contains a list of pointers to
server structs (called serverinfo) and it keeps a total on how many servers are
in the list.

Each server struct contains a list of pointers to groupstructs (called
groupinfo) and number wich indicates the total number of groups to be visited.

Each groupstruct contains a list of pointers to a (so called) postingstruct
A posting is a group of files with a common destination, for example all the 
parts in a multipart binary posting. If they are all in a single posting
they will be concatenated and sent to a destination (stdout, file, pipe).

After the sanity checks the actual processing starts. It uses the functions
in visits.c . For sending and recieving NNTP stuff, the functions in nntp.c are
used. And finally tcp.c contains the low level network code.

The parsing code is currently a bit ugly and inelegant.

MAKEFILE

The makefile contains a 'dist' target. It wil create a .tar.gz file in the 
parent directory. It will only work if the newsleech directory has the right 
name. It will overwrite .tar.gz and .tar.bz2 files with the same name. 

The version of newsleech is defined in the Makefile.

NEWS

Please see the file NEWS for the latest news.