something to do

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

something to do

rvjansen
So, you are at home in the crisis. NetFlix is contemplating to remove the ‘are you still there?’ check. People are switching off the standby mode of their TV sets. You might do something with that brain and all your experience. NetRexx needs you!

NetRexx is unique in that it is your favourite programming language, and in that it is very easy to build. Other Rexx dialects need you at least to install a C(++) compiler, a build system, some libraries (with different names on each platform) and other things you encounter after trying several times.

NetRexx, however, requires a working Java 8 installation(, and git to check out the source). On macOS, Linux, and Windows. You heard that right, the build procedure is the same on Windows. This is because NetRexx is (entirely) written in NetRexx, which means you only need a previous version of NetRexx to compile it.

You can find NetRexx source on SourceForge in our git repository. You can do a read-only checkout over https, but if you really want to help out, do a checkout over ssh; then you can collaborate on our favourite piece of software. There is something about collaboration: I have friends I have never seen, but they are close to my heart. Some friends I did meet, and it was through (Net)Rexx only.

I will post detailed instructions next.

best regards,

René.
_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: something to do

Jeff Hennick-3

Let me second René's thoughts.

[I am sitting in an excellent location, if it is needed to be shut in.  One of the four walls I am staring at is glass and looks over a large estuary.]

Specifically the njPipes portion has much of its structure in small individual NetRexx programs that can be worked on without any chance of damaging anything else.  While we have many of the ones in CMS Pipelines, there are also many still missing.  Any of these make an easy "one person project."  And these are well specified in the IBM publications.

If anyone wishes to work on this, jump in!  Or, I can suggest one for you, together with pointers on others to inspect for the structure -- there is very little boilerplate needed to fit into the pipe. 

Stay well,

Jeff

On 3/20/2020 5:26 AM, René Jansen wrote:
So, you are at home in the crisis. NetFlix is contemplating to remove the ‘are you still there?’ check. People are switching off the standby mode of their TV sets. You might do something with that brain and all your experience. NetRexx needs you!

NetRexx is unique in that it is your favourite programming language, and in that it is very easy to build. Other Rexx dialects need you at least to install a C(++) compiler, a build system, some libraries (with different names on each platform) and other things you encounter after trying several times.

NetRexx, however, requires a working Java 8 installation(, and git to check out the source). On macOS, Linux, and Windows. You heard that right, the build procedure is the same on Windows. This is because NetRexx is (entirely) written in NetRexx, which means you only need a previous version of NetRexx to compile it.

You can find NetRexx source on SourceForge in our git repository. You can do a read-only checkout over https, but if you really want to help out, do a checkout over ssh; then you can collaborate on our favourite piece of software. There is something about collaboration: I have friends I have never seen, but they are close to my heart. Some friends I did meet, and it was through (Net)Rexx only.

I will post detailed instructions next.

best regards,

René.
_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: something to do

Jason Martin
In reply to this post by rvjansen
Do not need previous installed? It already comes with checkout right?

On 2020-03-20 05:26, René Jansen wrote:
> which means you only need a previous version of NetRexx to compile it.

_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: something to do

rvjansen
Yes, I forgot to tell that - the previous version you need, is already included in the checkout from the repository! And set up correctly.

> On 20 Mar 2020, at 12:51, Jason Martin <[hidden email]> wrote:
>
> Do not need previous installed? It already comes with checkout right?
>
> On 2020-03-20 05:26, René Jansen wrote:
>> which means you only need a previous version of NetRexx to compile it.
>
> _______________________________________________
> Ibm-netrexx mailing list
> [hidden email]
> Online Archive : https://urldefense.proofpoint.com/v2/url?u=http-3A__ibm-2Dnetrexx.215625.n3.nabble.com_&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=_6rXNpPJ1fYV-3bV1za02NiR4PUelvicfHXwtnTXpXE&m=aiZvgkzJ2TEnOYgo182KpAvL9ycYg0hHk-J-1EbbUXA&s=A4G4UKtA7ZVYKgTFxi7wP_ILK21Acs8BIRQrllDGQVQ&e= 
>

_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Pipes and HOSTBYNAME and Other Things [ was Re: something to do ]

ColinK
In reply to this post by Jeff Hennick-3
Hi Jeff

I thought I would let you know that as part of my pipelines education, so-to-speak, I decided to tackle one of the missing CMS filters in njpipes, namely HOSTBYNAME.  I figure if I can get this working then the other HOST* related filters (HOSTBYADDR, HOSTNAME and HOSTID) should hopefully follow with relatively minor work being required.  This note is a bit long and I hope it formats OK on the newsgroup.

I do have a version that is working, or appears to be working.  The plumbing is a bit roughed in at the moment (humour intended)  and needs some documentation and test cases but on the surface it looks promising.

Z:\NetRexx-3.08-GA\bin>pipe "literal www.shaw.ca | hostbyname | console"
23.6.246.208

Z:\NetRexx-3.08-GA\bin>pipe "literal dns.google | hostbyname | console"
8.8.4.4 8.8.8.8

Z:\NetRexx-3.08-GA\bin>pipe "literal bogus.bogus | hostbyname | console"
Unable to determine IP address for bogus.bogus

As part of doing this I have some questions and observations.

CMS HOSTBYNAME Examples

Do you have access, or know of someone who does have access, to a VM/CMS system?  In order to verify that my filter is on the right track I am looking for the CMS console output for the following

pipe literal dns.google | hostbyname | console

pipe literal ipv6.google.com | hostbyname | console

The former domain name is an example that has multiple A and AAAA records and the latter is an example of a pure AAAA record.  I need to know how CMS HOSTBYNAME handles multiple IP addresses for a host.  Also, I am noticing that Java’s HOSTBYNAME implementation seems to have issues where IPv6 addresses are concerned and am curious as to how CMS Pipes handles this situation.

Enhancements

The CMS version of HOSTBYNAME has no optional parameters.  I have a few ideas of optional parameters that a user may find useful.  Would it be acceptable to include these in my code but have the filter without any parameters work the same as the CMS version?  Or is the goal to mimic the CMS version precisely; thus no optional parameters should be offered?

Acceptance Process

Assuming that I manage to get my code working what is the process that one follows to get the code incorporated into the base?  At a minimum someone with more NetRexx/Java and Pipelines skills than me needs to review my work.  There are probably things I have missed or not considered that would affect the quality of the filter.  Or, it is possible that what I have created is nowhere good enough for general distribution.  That is an OK response too – I have learned a lot in doing this and quite enjoyed the process.


Anomaly With INSERT Filter

During the building of the filter HOSTBYNAME I tried the following example found in the CMS v6.4 Pipelines manual (p 438)

|     Use the secondary output stream to handle hostnames that could not be resolved:
|     pipe (end ?) literal www.mvs.ibm.com | h: hostbyname | cons ? h: | ...
|     ... insert ,Bad: , | console
|     ►Bad: www.mvs.ibm.com 2016 EHOSTNOTFOUND Host not found in SITEINFO file
|     ►Ready;

njpipes version:

Z:\NetRexx-3.08-GA\bin>pipe "(end ?) literal xxxdns.google | h: hostbyname2 | console ? h: | insert ,BAD , | console"
insert_4 Error - insert found invalid parm  rc=11
o=null
RC=11

In looking at the insert.nrx filter code it turns out there is no default for the BEFORE/AFTER/inputRange selections.  The CMS version of INSERT seems to make the default BEFORE.  

If I add BEFORE in my pipe then the error message goes away.

Z:\NetRexx-3.08-GA\bin>pipe "(end ?) literal xxxdns.google | h: hostbyname2 | console ? h: | insert ,BAD , BEFORE | console"
o=null
UnknownHostException

The UnknownHostException is what I expected as I had not yet added code to handle.

Unfortunately the CMS manual provides what I think is a cryptic answer to the question as to what the default should be.  CMS v6.4 Pipelines manual p. 445

Operation: When the input range is not present in the record, the range used is the first
or the last column of the record, depending on whether the first part of the inputRange is relative to the beginning or the end of the record.

In my case, and the CMS case, inputRange is not specified but I cannot decipher what it is saying.  That has got to be one of the most confusing statements I have ever read.  If inputRange is not present then how can there be a “first part of the inputRange”?   In the CMS example the default seems to imply BEFORE.

I am thinking that the njpipes version needs to have a default coded to make it function like the CMS version.  I am just not certain as to what that default should be.

If this is something that should be addressed then should I raise a bug issue and if so where?  Your thoughts would be appreciated.

Documentation Suggestion

In the documentation for Pipes (Pipelines for NetRexx QuickStart Guide v3.08) it seems to me that “stage” and “filter” are used interchangeably but I am worried that I may be missing some subtle distinction between the two.  So my question is “Is there a distinction between the two or can they be used interchangeably?” 

If there is a distinction then perhaps this can be made clearer in section 2.1 “What is a Pipeline” or section 2.2 “Stage”.

Assuming they can be used interchangeably then may I suggest renaming section 10: Other Stages   to  10: Other Filters   to be consistent with the other section headers that use the word “Filters”.

I know that this is being nit-picky but it looks more consistent in my eyes at least.  I am probably being a bit anal here (or as one of my polite colleagues once said: “structured” :-) ).

Non CMS Filters

While I understand the goal is to mimic the CMS filters in the NetRexx implementation, is there some central location where people who develop non-CMS filters can make their efforts known? 

Thanks
ColinK





On 2020-03-20 04:30, Jeff Hennick wrote:

Let me second René's thoughts.

[I am sitting in an excellent location, if it is needed to be shut in.  One of the four walls I am staring at is glass and looks over a large estuary.]

Specifically the njPipes portion has much of its structure in small individual NetRexx programs that can be worked on without any chance of damaging anything else.  While we have many of the ones in CMS Pipelines, there are also many still missing.  Any of these make an easy "one person project."  And these are well specified in the IBM publications.

If anyone wishes to work on this, jump in!  Or, I can suggest one for you, together with pointers on others to inspect for the structure -- there is very little boilerplate needed to fit into the pipe. 

Stay well,

Jeff

On 3/20/2020 5:26 AM, René Jansen wrote:
So, you are at home in the crisis. NetFlix is contemplating to remove the ‘are you still there?’ check. People are switching off the standby mode of their TV sets. You might do something with that brain and all your experience. NetRexx needs you!

NetRexx is unique in that it is your favourite programming language, and in that it is very easy to build. Other Rexx dialects need you at least to install a C(++) compiler, a build system, some libraries (with different names on each platform) and other things you encounter after trying several times.

NetRexx, however, requires a working Java 8 installation(, and git to check out the source). On macOS, Linux, and Windows. You heard that right, the build procedure is the same on Windows. This is because NetRexx is (entirely) written in NetRexx, which means you only need a previous version of NetRexx to compile it.

You can find NetRexx source on SourceForge in our git repository. You can do a read-only checkout over https, but if you really want to help out, do a checkout over ssh; then you can collaborate on our favourite piece of software. There is something about collaboration: I have friends I have never seen, but they are close to my heart. Some friends I did meet, and it was through (Net)Rexx only.

I will post detailed instructions next.

best regards,

René.
_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes and HOSTBYNAME and Other Things [ was Re: something to do ]

Jeff Hennick-3

Colin,

Thanks for your help!

One question that I can take care of now is the INSERT without BEFORE/AFTER.  I had noticed that bug in my testing and have put in the fix.  It will be in this next  NetRexx 3.09.  (Thanks for the prompt, however.  In my other work, I had forgotten to submit it until today.)

On the "what does CMS do?" question, I, in the middle of all else, am trying to get a local emulator working, with great help from Rene, but am not there yet. -- I have had similar questions from the IBM documentation.  And the only real answer is, as you are asking, to "Ask THE expert:" a running machine.

On the extensions to the IBM stages, I come down on the "Yes" side to "complete" the design of the stage. Here is an example:

/** drop

            +-FIRST-+  +-1---------+
  >>--DROP--+-------+--+-----------+--+----------+-----------------------><
            +-LAST--+  +-number-(2)+  +-BYTES-(1)+
                       +-*---------+

  (1) Not CMS
  (2) CMS: must be positive.
      NJPipes: negative reverses FIRST/LAST, so DROP FIRST -3 is the same as DROP LAST 3.

*/

(I don't like "except"s in specifications.  And njPipes works in a world of byte-oriented files.)

But, and this is a balancing act, I personally like pipe stages to "do only one thing, but do it very well."  I'd rather have and use 2 or 3 small stages than 1 overly complex stage.  It should not take many tests to "prove" a stage is working.  My current rant target is the newest IBM SPECS stage.  They have added a whole new "scripting language" into the middle of it -- not even using Rexx, the King of Scripting Languages!  I am up to about 50 separate simple tests for SPECS and am not done yet.  And it has lots of "moving parts" that interact with each other: more tests!  (LOOKUP is another example.)

NOTE that this njPipes version of INSERT is not up to the documented CMS version, which adds an "inputRange".  Which you have the question about its documentation.  First, I think they mean outputRange, but would have to "ask the expert."  If I am right, then the "depending" would be on the BEFORE or AFTER choice.  As I read it, the inputRange replaces the BEFORE or AFTER, but the diagram doesn't say that.

Stage vs Filter.  My understanding is they are all Stages.  A Filter is a type of stage that has a single input and a single (or two) output that either selects some records and not others, or that changes each record.  LOCATE filters its input, so does INSERT.   LOOKUP would not be a filter.  I don't know how I'd categorize HOSTNAME.  It makes no real difference if a stage is called a filter, it is used and written the same way.  I imagine that Filters tend to be simpler stages  (Now, why we have "stages" in plumbing, I have no idea.)

Jeff

On 3/20/2020 11:49 AM, ColinK wrote:
Hi Jeff

I thought I would let you know that as part of my pipelines education, so-to-speak, I decided to tackle one of the missing CMS filters in njpipes, namely HOSTBYNAME.  I figure if I can get this working then the other HOST* related filters (HOSTBYADDR, HOSTNAME and HOSTID) should hopefully follow with relatively minor work being required.  This note is a bit long and I hope it formats OK on the newsgroup.

I do have a version that is working, or appears to be working.  The plumbing is a bit roughed in at the moment (humour intended)  and needs some documentation and test cases but on the surface it looks promising.

Z:\NetRexx-3.08-GA\bin>pipe "literal www.shaw.ca | hostbyname | console"
23.6.246.208

Z:\NetRexx-3.08-GA\bin>pipe "literal dns.google | hostbyname | console"
8.8.4.4 8.8.8.8

Z:\NetRexx-3.08-GA\bin>pipe "literal bogus.bogus | hostbyname | console"
Unable to determine IP address for bogus.bogus

As part of doing this I have some questions and observations.

CMS HOSTBYNAME Examples

Do you have access, or know of someone who does have access, to a VM/CMS system?  In order to verify that my filter is on the right track I am looking for the CMS console output for the following

pipe literal dns.google | hostbyname | console

pipe literal ipv6.google.com | hostbyname | console

The former domain name is an example that has multiple A and AAAA records and the latter is an example of a pure AAAA record.  I need to know how CMS HOSTBYNAME handles multiple IP addresses for a host.  Also, I am noticing that Java’s HOSTBYNAME implementation seems to have issues where IPv6 addresses are concerned and am curious as to how CMS Pipes handles this situation.

Enhancements

The CMS version of HOSTBYNAME has no optional parameters.  I have a few ideas of optional parameters that a user may find useful.  Would it be acceptable to include these in my code but have the filter without any parameters work the same as the CMS version?  Or is the goal to mimic the CMS version precisely; thus no optional parameters should be offered?

Acceptance Process

Assuming that I manage to get my code working what is the process that one follows to get the code incorporated into the base?  At a minimum someone with more NetRexx/Java and Pipelines skills than me needs to review my work.  There are probably things I have missed or not considered that would affect the quality of the filter.  Or, it is possible that what I have created is nowhere good enough for general distribution.  That is an OK response too – I have learned a lot in doing this and quite enjoyed the process.


Anomaly With INSERT Filter

During the building of the filter HOSTBYNAME I tried the following example found in the CMS v6.4 Pipelines manual (p 438)

|     Use the secondary output stream to handle hostnames that could not be resolved:
|     pipe (end ?) literal www.mvs.ibm.com | h: hostbyname | cons ? h: | ...
|     ... insert ,Bad: , | console
|     ►Bad: www.mvs.ibm.com 2016 EHOSTNOTFOUND Host not found in SITEINFO file
|     ►Ready;

njpipes version:

Z:\NetRexx-3.08-GA\bin>pipe "(end ?) literal xxxdns.google | h: hostbyname2 | console ? h: | insert ,BAD , | console"
insert_4 Error - insert found invalid parm  rc=11
o=null
RC=11

In looking at the insert.nrx filter code it turns out there is no default for the BEFORE/AFTER/inputRange selections.  The CMS version of INSERT seems to make the default BEFORE.  

If I add BEFORE in my pipe then the error message goes away.

Z:\NetRexx-3.08-GA\bin>pipe "(end ?) literal xxxdns.google | h: hostbyname2 | console ? h: | insert ,BAD , BEFORE | console"
o=null
UnknownHostException

The UnknownHostException is what I expected as I had not yet added code to handle.

Unfortunately the CMS manual provides what I think is a cryptic answer to the question as to what the default should be.  CMS v6.4 Pipelines manual p. 445

Operation: When the input range is not present in the record, the range used is the first
or the last column of the record, depending on whether the first part of the inputRange is relative to the beginning or the end of the record.

In my case, and the CMS case, inputRange is not specified but I cannot decipher what it is saying.  That has got to be one of the most confusing statements I have ever read.  If inputRange is not present then how can there be a “first part of the inputRange”?   In the CMS example the default seems to imply BEFORE.

I am thinking that the njpipes version needs to have a default coded to make it function like the CMS version.  I am just not certain as to what that default should be.

If this is something that should be addressed then should I raise a bug issue and if so where?  Your thoughts would be appreciated.

Documentation Suggestion

In the documentation for Pipes (Pipelines for NetRexx QuickStart Guide v3.08) it seems to me that “stage” and “filter” are used interchangeably but I am worried that I may be missing some subtle distinction between the two.  So my question is “Is there a distinction between the two or can they be used interchangeably?” 

If there is a distinction then perhaps this can be made clearer in section 2.1 “What is a Pipeline” or section 2.2 “Stage”.

Assuming they can be used interchangeably then may I suggest renaming section 10: Other Stages   to  10: Other Filters   to be consistent with the other section headers that use the word “Filters”.

I know that this is being nit-picky but it looks more consistent in my eyes at least.  I am probably being a bit anal here (or as one of my polite colleagues once said: “structured” :-) ).

Non CMS Filters

While I understand the goal is to mimic the CMS filters in the NetRexx implementation, is there some central location where people who develop non-CMS filters can make their efforts known? 

Thanks
ColinK





On 2020-03-20 04:30, Jeff Hennick wrote:

Let me second René's thoughts.

[I am sitting in an excellent location, if it is needed to be shut in.  One of the four walls I am staring at is glass and looks over a large estuary.]

Specifically the njPipes portion has much of its structure in small individual NetRexx programs that can be worked on without any chance of damaging anything else.  While we have many of the ones in CMS Pipelines, there are also many still missing.  Any of these make an easy "one person project."  And these are well specified in the IBM publications.

If anyone wishes to work on this, jump in!  Or, I can suggest one for you, together with pointers on others to inspect for the structure -- there is very little boilerplate needed to fit into the pipe. 

Stay well,

Jeff

On 3/20/2020 5:26 AM, René Jansen wrote:
So, you are at home in the crisis. NetFlix is contemplating to remove the ‘are you still there?’ check. People are switching off the standby mode of their TV sets. You might do something with that brain and all your experience. NetRexx needs you!

NetRexx is unique in that it is your favourite programming language, and in that it is very easy to build. Other Rexx dialects need you at least to install a C(++) compiler, a build system, some libraries (with different names on each platform) and other things you encounter after trying several times.

NetRexx, however, requires a working Java 8 installation(, and git to check out the source). On macOS, Linux, and Windows. You heard that right, the build procedure is the same on Windows. This is because NetRexx is (entirely) written in NetRexx, which means you only need a previous version of NetRexx to compile it.

You can find NetRexx source on SourceForge in our git repository. You can do a read-only checkout over https, but if you really want to help out, do a checkout over ssh; then you can collaborate on our favourite piece of software. There is something about collaboration: I have friends I have never seen, but they are close to my heart. Some friends I did meet, and it was through (Net)Rexx only.

I will post detailed instructions next.

best regards,

René.
_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes and HOSTBYNAME and Other Things [ was Re: something to do ]

rvjansen
In reply to this post by ColinK
Hi Colin,

very happy to see your email. Sorry for the late reaction, am working through last week in reverse order and yours unfortunately is one of the last.


As part of doing this I have some questions and observations.

CMS HOSTBYNAME Examples

Do you have access, or know of someone who does have access, to a VM/CMS system?  In order to verify that my filter is on the right track I am looking for the CMS console output for the following

pipe literal dns.google | hostbyname | console

pipe literal ipv6.google.com | hostbyname | console



 pipe literal google.com | hostbyname | cons                                                                          
172.217.8.142                                                                                                         
Ready; T=0.05/0.07 10:41:59                                                                                           
 pipe literal ipv6.google.com | hostbyname | cons                                                                     
PIPTCR1142E Unable to resolve ipv6.google.com (RXSOCKET error 2011 EDOMAINSERVERFAILURE Domain name server failure).  
PIPMSG003I ... Issued from stage 2 of pipeline 1.                                                                     
PIPMSG001I ... Running "hostbyname".                                                                                  
Ready(01142); T=0.04/0.07 10:43:01                                                                                    
                                                                                                                      
Note that this is from a machine in South America. I get another IP from the Netherlands. 

ipv6.google.com is not pingable from here either. Let me know if I need to run more tests.


The former domain name is an example that has multiple A and AAAA records and the latter is an example of a pure AAAA record.  I need to know how CMS HOSTBYNAME handles multiple IP addresses for a host.  Also, I am noticing that Java’s HOSTBYNAME implementation seems to have issues where IPv6 addresses are concerned and am curious as to how CMS Pipes handles this situation.

Enhancements

The CMS version of HOSTBYNAME has no optional parameters.  I have a few ideas of optional parameters that a user may find useful.  Would it be acceptable to include these in my code but have the filter without any parameters work the same as the CMS version?  Or is the goal to mimic the CMS version precisely; thus no optional parameters should be offered?

Addtional parameters are welcome, they are not in the way of anyone knowing the CMS version but just might help out someone in accomplishing a specific goal. We try to document the differences, though.


Acceptance Process

Assuming that I manage to get my code working what is the process that one follows to get the code incorporated into the base?  At a minimum someone with more NetRexx/Java and Pipelines skills than me needs to review my work.  There are probably things I have missed or not considered that would affect the quality of the filter.  Or, it is possible that what I have created is nowhere good enough for general distribution.  That is an OK response too – I have learned a lot in doing this and quite enjoyed the process.



As I see it, work on stages almost never steps on something else, so if you send me you SourceForge ID, I’ll add you as a committer right away. Your help is very much appreciated.
Of course Jeff and I will have a look at your code and will comment and support if needed. Please deliver test cases with all added functionality, that will make it easier for us all.

You can also email your contributions to me and I will check them in, if for some reason you don’t want to use git. 


best regards,

René.

_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes and HOSTBYNAME and Other Things [ was Re: something to do ]

rvjansen
In reply to this post by ColinK
Colin,

On 20 Mar 2020, at 16:49, ColinK <[hidden email]> wrote:

Documentation Suggestion

In the documentation for Pipes (Pipelines for NetRexx QuickStart Guide v3.08) it seems to me that “stage” and “filter” are used interchangeably but I am worried that I may be missing some subtle distinction between the two.  So my question is “Is there a distinction between the two or can they be used interchangeably?”  

If there is a distinction then perhaps this can be made clearer in section 2.1 “What is a Pipeline” or section 2.2 “Stage”.


Yes, this is all my fault. I never could find the time to do this properly. You are very welcome to improve on it, most preferably by editing the tex source in documentation/njpipes. If you use Docker, I have an image that has everything to build the documentation — which is more complex than building the NetRexx translator and Pipelines itself).


Assuming they can be used interchangeably then may I suggest renaming section 10: Other Stages   to  10: Other Filters   to be consistent with the other section headers that use the word “Filters”.

I know that this is being nit-picky but it looks more consistent in my eyes at least.  I am probably being a bit anal here (or as one of my polite colleagues once said: “structured” :-) ).

Nitpicky is good! It is the only way in our profession. I never could argue with ASMH (or I lost every argument with it).


Non CMS Filters

While I understand the goal is to mimic the CMS filters in the NetRexx implementation, is there some central location where people who develop non-CMS filters can make their efforts known?  

There is a special list for NetRexx Pipelines, be sure to subscribe to it when arranging your SourceForge ID. ( it is called [hidden email] (I was slightly nudged by people who are interested in NetRexx but not in CMS Pipelines )

René.


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes and HOSTBYNAME and Other Things [ was Re: something to do ]

ColinK
In reply to this post by rvjansen
Hi René

Thanks so much for getting the sample VM/CMS HOSTBYNAME.

I was actually hoping for "dns.google" and not "google.com".  What this does show is that VM/CMS HOSTBYNAME behaves just as badly as Java's getByName when it comes to handling IPv6 addresses.

Z:\NetRexx-3.08-GA\bin>pipe "literal google.com | hostbyname | console"
216.58.193.78
Z:\NetRexx-3.08-GA\bin>nslookup google.com
Server:  dns.google
Address:  8.8.8.8

Non-authoritative answer:
Name:    google.com
Addresses:  2607:f8b0:400a:800::200e
          172.217.14.238

Z:\NetRexx-3.08-GA\bin>pipe "literal ipv6.google.com | hostbyname | console"
Unable to determine IP address for ipv6.google.com

Z:\NetRexx-3.08-GA\bin>nslookup ipv6.google.com 1.1.1.1
Server:  one.one.one.one
Address:  1.1.1.1

Non-authoritative answer:
Name:    ipv6.l.google.com
Address:  2607:f8b0:400a:804::200e
Aliases:  ipv6.google.com


Z:\NetRexx-3.08-GA\bin>nslookup ipv6.google.com 8.8.8.8
Server:  dns.google
Address:  8.8.8.8

Non-authoritative answer:
Name:    ipv6.l.google.com
Address:  2607:f8b0:400a:801::200e
Aliases:  ipv6.google.com

It would be nice if you could repeat the test you did for me using "dns.google".  This would give me an example of how CMS HOSTBYNAME handles multiple IPv4 addresses in a response.

I am not sure if I should replace Java's getByName with an nslookup equivalent (or make it an option for the user to choose which method to use for name resolution; with Java being the default so it works just like CMS).  Or perhaps leave as is and create a standalone NSLOOKUP version instead.

Any thoughts on the preferable way to proceed?

Thanks
Colin




On 2020-03-22 08:11, René Jansen wrote:
Hi Colin,

very happy to see your email. Sorry for the late reaction, am working through last week in reverse order and yours unfortunately is one of the last.


As part of doing this I have some questions and observations.

CMS HOSTBYNAME Examples

Do you have access, or know of someone who does have access, to a VM/CMS system?  In order to verify that my filter is on the right track I am looking for the CMS console output for the following

pipe literal dns.google | hostbyname | console

pipe literal ipv6.google.com | hostbyname | console



 pipe literal google.com | hostbyname | cons                                                                          
172.217.8.142                                                                                                         
Ready; T=0.05/0.07 10:41:59                                                                                           
 pipe literal ipv6.google.com | hostbyname | cons                                                                     
PIPTCR1142E Unable to resolve ipv6.google.com (RXSOCKET error 2011 EDOMAINSERVERFAILURE Domain name server failure).  
PIPMSG003I ... Issued from stage 2 of pipeline 1.                                                                     
PIPMSG001I ... Running "hostbyname".                                                                                  
Ready(01142); T=0.04/0.07 10:43:01                                                                                    
                                                                                                                      
Note that this is from a machine in South America. I get another IP from the Netherlands. 

ipv6.google.com is not pingable from here either. Let me know if I need to run more tests.


The former domain name is an example that has multiple A and AAAA records and the latter is an example of a pure AAAA record.  I need to know how CMS HOSTBYNAME handles multiple IP addresses for a host.  Also, I am noticing that Java’s HOSTBYNAME implementation seems to have issues where IPv6 addresses are concerned and am curious as to how CMS Pipes handles this situation.

Enhancements

The CMS version of HOSTBYNAME has no optional parameters.  I have a few ideas of optional parameters that a user may find useful.  Would it be acceptable to include these in my code but have the filter without any parameters work the same as the CMS version?  Or is the goal to mimic the CMS version precisely; thus no optional parameters should be offered?

Addtional parameters are welcome, they are not in the way of anyone knowing the CMS version but just might help out someone in accomplishing a specific goal. We try to document the differences, though.


Acceptance Process

Assuming that I manage to get my code working what is the process that one follows to get the code incorporated into the base?  At a minimum someone with more NetRexx/Java and Pipelines skills than me needs to review my work.  There are probably things I have missed or not considered that would affect the quality of the filter.  Or, it is possible that what I have created is nowhere good enough for general distribution.  That is an OK response too – I have learned a lot in doing this and quite enjoyed the process.



As I see it, work on stages almost never steps on something else, so if you send me you SourceForge ID, I’ll add you as a committer right away. Your help is very much appreciated.
Of course Jeff and I will have a look at your code and will comment and support if needed. Please deliver test cases with all added functionality, that will make it easier for us all.

You can also email your contributions to me and I will check them in, if for some reason you don’t want to use git. 


best regards,

René.

_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes and HOSTBYNAME and Other Things [ was Re: something to do ]

rvjansen
Hi Colin,

This is what CMS does:

 pipe literal dns.google.com | hostbyname | cons    
8.8.4.4 8.8.8.8                                    
Ready; T=0.07/0.09 18:11:30                        
                                                   

>
>
> It would be nice if you could repeat the test you did for me using "dns.google".  This would give me an example of how CMS HOSTBYNAME handles multiple IPv4 addresses in a response.
>
> I am not sure if I should replace Java's getByName with an nslookup equivalent (or make it an option for the user to choose which method to use for name resolution; with Java being the default so it works just like CMS).  Or perhaps leave as is and create a standalone NSLOOKUP version instead.
>
> Any thoughts on the preferable way to proceed?
>


I would prefer it to be Java because all native solutions always complicate things. If it does what CMS does, the better.

I am a bit behind with the release, so the stage could still make 3.09!

best regards,

René.

_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes and HOSTBYNAME and Other Things [ was Re: something to do ]

ColinK
Hi Renee

That is excellent!  Thanks so much for getting me that example output; it helps greatly.

Also, thank you for your thoughts and guidance on how to proceed.  At the rate I am getting time to work on it I very much doubt it will make the v3.09 release.  I had visions that this Covid-19 isolating stuff would give me lots of time to work on it; but alas not so.

Cheers,
Colin

On 2020-03-23 15:14, René Jansen wrote:
Hi Colin,

This is what CMS does:

 pipe literal dns.google.com | hostbyname | cons    
8.8.4.4 8.8.8.8                                     
Ready; T=0.07/0.09 18:11:30                         
                                                    


It would be nice if you could repeat the test you did for me using "dns.google".  This would give me an example of how CMS HOSTBYNAME handles multiple IPv4 addresses in a response.

I am not sure if I should replace Java's getByName with an nslookup equivalent (or make it an option for the user to choose which method to use for name resolution; with Java being the default so it works just like CMS).  Or perhaps leave as is and create a standalone NSLOOKUP version instead.

Any thoughts on the preferable way to proceed?


I would prefer it to be Java because all native solutions always complicate things. If it does what CMS does, the better.

I am a bit behind with the release, so the stage could still make 3.09!

best regards,

René.

_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/