pipes for netrexx - example 2: a pipe filter in NetRexx

[rvjansen]

So we have seen in the previous example that it is not too hard to make a simple pipeline out of things called 'device drivers' (such as command, for OS commands, diskr for reading files on disk, and literal, for inserting literal strings into a pipeline (actually you have seen only command, and in this episode we are only seeing diskr (for diskread), which is called '<' on CMS), filters and sinks.

We received some comments, here and on the CMS-Pipelines list, that the first example was too easy and more representative for Unix pipes than for VM, which can do more involved multistream stuff. That is true, but I would like to start with the easier examples so everybody is still comfortable. So please hold your horses while I show you something that is also really cool. And please try the things we showed with the spec stages in a Unix pipe: you'll probably need awk to do that and it will not be as clean as the specs examples, so it is not that easy; writing a Unix filter in C is even more work.

One of the most appealing characteristics of piping on CMS and TSO is that you can make your own filters in Rexx, and that it is really simple. Of course, this also works in NetRexx and its Pipelines implementation. Imagine, for the sake of argument, that you have an assignment to quickly prefix the lines in a file with their lengths (in decimal, but also in hex). On CMS, that would look like:

FILE: LENGTH REXX

Signal on novalue
numeric digits 12

signal on error

do forever
'peekto line'
/* Process line here */
l=length(line)
'output' l d2x(l) line
'readto'
end
error: exit RC*(wordpos(RC, '8 12')=0)

And you would need to remember to call your filetype REXX instead of EXEC. Just sayin’ because that cost me two minutes of feeling less intelligent than usual. The ‘peekto’ reads the input but does not actually commit the read yet, so you can read it one more time with knowledge about the contents. The ‘output’ pushes its argument back into the pipeline. The ‘readto’ reads and commits the read so the line is really processed and we can go to the next one.

In NetRexx, that would be about the same, mutatis mutandis the object oriented stuff you don't have in Classic Rexx. Here peekto(), readto() and output() are method calls on the ’stage’ object.

FILE: length.nrx

import org.netrexx.njpipes.pipes.

class length extends stage

method run()
  do
    loop forever
      line = rexx peekto()

      l = line.length

      output(l l.d2x line)
      readto()
    end
  catch StageError
    rc = rc()
  end
  exit(rc*(rc<>12))

So that would look fairly familiar, and admittedly, a bit easier for us already well versed in NetRexx.

We can test this by building a pipeline and running the filter on its own source:

pipe "(testl sep |) diskr length.nrx | length | console"

java testl

If you have a CMS handy, that would be:

PIPE < LENGTH REXX | LENGTH | CONSOLE

on the first, Classic Rexx version of the filter. Note that the < on CMS means the same as the diskr on the NetRexx version.

The next example is going to be: how to use a pipe in your NetRexx program, and how to use the pipe compiler to have your pipeline specification and the filter definition in one sourcefile. 

best regards,

Rene.

p.s. These examples work on NetRexx 3.07. There is a version 3.08 coming soon that fixes some of the pipes examples and some of the included stages, for example specs with a c2x formatter. If you want to use it, check it out of the git and build it yourself, it is not hard and only requires git, and a java 8 installation (jre would be enough).

_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

[jlturriff]

On 2019-03-24 23:02:56 René Jansen wrote:

> So we have seen in the previous example that it is not too hard to make a
> simple pipeline out of things called 'device drivers' (such as command, for
> OS commands, diskr for reading files on disk, and literal, for inserting
> literal strings into a pipeline (actually you have seen only command, and
> in this episode we are only seeing diskr (for diskread), which is called
> '<' on CMS), filters and sinks. We received some comments, here and on the
> CMS-Pipelines list, that the first example was too easy and more
> representative for Unix pipes than for VM, which can do more involved
> multistream stuff. That is true, but I would like to start with the easier
> examples so everybody is still comfortable. So please hold your horses
> while I show you something that is also really cool. And please try the
> things we showed with the spec stages in a Unix pipe: you'll probably need
> awk to do that and it will not be as clean as the specs examples, so it is
> not that easy; writing a Unix filter in C is even more work. One of the
> most appealing characteristics of piping on CMS and TSO is that you can
> make your own filters in Rexx, and that it is really simple. Of course,
> this also works in NetRexx and its Pipelines implementation. Imagine, for
> the sake of argument, that you have an assignment to quickly prefix the
> lines in a file with their lengths (in decimal, but also in hex). On CMS,
> that would look like: FILE: LENGTH REXX
> Signal on novalue
> numeric digits 12
>
> signal on error
>
> do forever
> 'peekto line'
> /* Process line here */
> l=length(line)
> 'output' l d2x(l) line
> 'readto'
> end
> error: exit RC*(wordpos(RC, '8 12')=0)
> And you would need to remember to call your filetype REXX instead of EXEC.
> Just sayin’ because that cost me two minutes of feeling less intelligent
> than usual. The ‘peekto’ reads the input but does not actually commit the
> read yet, so you can read it one more time with knowledge about the
> contents. The ‘output’ pushes its argument back into the pipeline. The
> ‘readto’ reads and commits the read so the line is really processed and we
> can go to the next one. In NetRexx, that would be about the same, mutatis
> mutandis the object oriented stuff you don't have in Classic Rexx. Here
> peekto(), readto() and output() are method calls on the ’stage’ object.
> FILE: length.nrx
> import org.netrexx.njpipes.pipes.
> class length extends stage
> method run()
>   do
>     loop forever
>       line = rexx peekto()
>       l = line.length
>       output(l l.d2x line)
>       readto()
>     end
>   catch StageError
>     rc = rc()
>   end
>   exit(rc*(rc<>12))
> So that would look fairly familiar, and admittedly, a bit easier for us
> already well versed in NetRexx. We can test this by building a pipeline and
> running the filter on its own source: pipe "(testl sep |) diskr length.nrx
> | length | console"
> java testl
> If you have a CMS handy, that would be:
> PIPE < LENGTH REXX | LENGTH | CONSOLE
> on the first, Classic Rexx version of the filter. Note that the < on CMS
> means the same as the diskr on the NetRexx version. The next example is
> going to be: how to use a pipe in your NetRexx program, and how to use the
> pipe compiler to have your pipeline specification and the filter definition
> in one sourcefile.
>
> best regards,
>
> Rene.
>
> p.s. These examples work on NetRexx 3.07. There is a version 3.08 coming
> soon that fixes some of the pipes examples and some of the included stages,
> for example specs with a c2x formatter. If you want to use it, check it out
> of the git and build it yourself, it is not hard and only requires git, and
> a java 8 installation (jre would be enough).

René,
        I have a suggestion for an example which is not too complicated to
understand, but shows the power of NetRexx Pipelines.  (It's something that
has made me wish for a long time for CMS Pipelines for Unix.)
        The Unix tree program shows a nicely formatted tree structure of any part of
the filesystem, either as a simple list of nodes or a list with added
information such as file size, user and group, timestamp, etc.  
Unfortunately, those additional items are shown to the right of the
indentations in the tree, making it difficult to compare various lines.  
NetRexx Pipelines can easily reposition these items to the left so that each
item appears in a uniform column for easy reading.  Here's my example:

Plain tree, easily readable:

> $ tree -d rexx/|head -n15 rexx/
> ├── dbus
> ├── dbus.rex -> dbus
> ├── gtk
> ├── gtk.rex -> gtk
> ├── habits
> ├── habits.rex -> habits
> ├── kdialog
> │   └── leslie
> │       └── bin
> │           └── rexx
> ├── kdialog.rex -> kdialog
> ├── ncurses
> │   ├── MouseEvents
> │   ├── test01

tree with additional info, hard to read:
> $ tree -pugsDd rexx/|head -n15 rexx/
> ├── [drwxr-xr-x leslie   users            164 Oct 15  2014]  dbus
> ├── [lrwxrwxrwx leslie   users              4 Jun  9  2015]  dbus.rex ->
dbus
> ├── [drwxr-xr-x leslie   users            320 Apr 18  2016]  gtk
> ├── [lrwxrwxrwx leslie   users              3 Jun  9  2015]  gtk.rex -> gtk
> ├── [drwxr-xr-x leslie   users            134 Oct 15  2014]  habits
> ├── [lrwxrwxrwx leslie   users              6 Jun  9  2015]  habits.rex ->
habits
> ├── [drwxr-xr-x leslie   users           2760 Oct 15  2014]  kdialog
> │   └── [drwxr-xr-x leslie   users              6 Aug 21  2014]  leslie
> │       └── [drwxr-xr-x leslie   users              8 Aug 21  2014]  bin
> │           └── [drwxr-xr-x leslie   users             70 Oct 15  2014]  
rexx
> ├── [lrwxrwxrwx leslie   users              7 Jun  9  2015]  kdialog.rex ->
kdialog
> ├── [drwxr-xr-x leslie   users            262 Feb 16  2015]  ncurses
> │   ├── [drwxr-xr-x leslie   users              8 Oct 15  2014]  MouseEvents
> │   ├── [drwxr-xr-x leslie   users              6 Oct 15  2014]  test01

tree with additional info, conceptually reformatted for readability by NetRexx
Pipelines:
> $ tree -pugsDd rexx/|head -n15 rexx/
> [drwxr-xr-x leslie   users            164 Oct 15  2014] ├── dbus
> [lrwxrwxrwx leslie   users              4 Jun  9  2015] ├── dbus.rex -> dbus
> [drwxr-xr-x leslie   users            320 Apr 18  2016] ├── gtk
> [lrwxrwxrwx leslie   users              3 Jun  9  2015] ├── gtk.rex -> gtk
> [drwxr-xr-x leslie   users            134 Oct 15  2014] ├── habits
> [lrwxrwxrwx leslie   users              6 Jun  9  2015] ├── habits.rex ->
habits
> [drwxr-xr-x leslie   users           2760 Oct 15  2014] ├── kdialog
> [drwxr-xr-x leslie   users              6 Aug 21  2014] │   └── leslie
> [drwxr-xr-x leslie   users              8 Aug 21  2014] │       └── bin
> [drwxr-xr-x leslie   users             70 Oct 15  2014] │           └── rexx
> [lrwxrwxrwx leslie   users              7 Jun  9  2015] ├── kdialog.rex ->
kdialog
> [drwxr-xr-x leslie   users            262 Feb 16  2015] ├── ncurses
> [drwxr-xr-x leslie   users              8 Oct 15  2014] │   ├── MouseEvents
> [drwxr-xr-x leslie   users              6 Oct 15  2014] │   ├── test01

        Also, since tree normally produces two lines of summary information after its
output, splitting those two lines off and joining them to the end of the
formatted lines would illustrate the advantages of use of a multi-stream
pipeline.

Leslie

_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/