Pipes Problem: LITERAL & CONS

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Pipes Problem: LITERAL & CONS

Jeff Hennick-3

I have set up alias classes for CONS, all the way out to CONSOLE.  And TERM to TERMINAL.  These work just fine individually.  Then I tried a compound test and got NULL-errors.

It is a problem that is wrapping me up in knots when I try to use two or more of them in a pipe with two or more LITERAL stages.

Here is when it goes bad, with pipe (ct ) literal Pipes is good! | cons ? literal done | cons ?, which I have in the file ct.njp .  You can see both literals and cons work and the data is displayed.  Then NULL pops up.

PS C:\Users\Jeff\documents\pipe tests> pipc ct pipe (ct ) literal Pipes is good! | cons ? literal done | cons ? PS C:\Users\Jeff\documents\pipe tests> java ct done Pipes is good! java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:88) at org.netrexx.njpipes.stages.cons.run(cons.java:130) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:76) at java.lang.Thread.run(Unknown Source) java.lang.NullPointerException Exiting due to uncaught Exception null at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:88) at org.netrexx.njpipes.stages.cons.run(cons.java:130) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:76) at java.lang.Thread.run(Unknown Source) Exception in thread "Thread-4" Exiting due to uncaught Exception null java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:93) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:108) at java.lang.Thread.run(Unknown Source) Exception in thread "Thread-2" java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:93) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:108) at java.lang.Thread.run(Unknown Source) PS C:\Users\Jeff\documents\pipe tests>

But by simply removing the second LITERAL stage (and changing the endpipe to endstage), it works:

PS C:\Users\Jeff\documents\pipe tests> pipc ct pipe (ct ) literal Pipes is good! | cons | cons ? PS C:\Users\Jeff\documents\pipe tests> java ct Pipes is good! Pipes is good! PS C:\Users\Jeff\documents\pipe tests>

At this point I don't even know if it is my messed up development environment.

Could someone please try that first pipe to see if it comes out clean?

Or, any help in pointing me to where the bug is would be most welcome.

Thank you.

Jeff Hennick


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes Problem: LITERAL & CONS

ColinK
HI Jeff

If I change cons to console it works fine for me.

Z:\NetRexx-3.08-GA\bin>pipe "(ct ) literal Pipes is good! | console ? literal done | console ?"
Pipes is good!
done

Z:\NetRexx-3.08-GA\bin>pipc ctnjp
pipe (ctnjp ) literal Pipes is good! | console ? literal done | console ?

Z:\NetRexx-3.08-GA\bin>java ctnjp
Pipes is good!
done

Hope this helps

Colin

On 2020-04-09 12:47, Jeff Hennick wrote:

I have set up alias classes for CONS, all the way out to CONSOLE.  And TERM to TERMINAL.  These work just fine individually.  Then I tried a compound test and got NULL-errors.

It is a problem that is wrapping me up in knots when I try to use two or more of them in a pipe with two or more LITERAL stages.

Here is when it goes bad, with pipe (ct ) literal Pipes is good! | cons ? literal done | cons ?, which I have in the file ct.njp .  You can see both literals and cons work and the data is displayed.  Then NULL pops up.

PS C:\Users\Jeff\documents\pipe tests> pipc ct pipe (ct ) literal Pipes is good! | cons ? literal done | cons ? PS C:\Users\Jeff\documents\pipe tests> java ct done Pipes is good! java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:88) at org.netrexx.njpipes.stages.cons.run(cons.java:130) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:76) at java.lang.Thread.run(Unknown Source) java.lang.NullPointerException Exiting due to uncaught Exception null at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:88) at org.netrexx.njpipes.stages.cons.run(cons.java:130) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:76) at java.lang.Thread.run(Unknown Source) Exception in thread "Thread-4" Exiting due to uncaught Exception null java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:93) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:108) at java.lang.Thread.run(Unknown Source) Exception in thread "Thread-2" java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:93) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:108) at java.lang.Thread.run(Unknown Source) PS C:\Users\Jeff\documents\pipe tests>

But by simply removing the second LITERAL stage (and changing the endpipe to endstage), it works:

PS C:\Users\Jeff\documents\pipe tests> pipc ct pipe (ct ) literal Pipes is good! | cons | cons ? PS C:\Users\Jeff\documents\pipe tests> java ct Pipes is good! Pipes is good! PS C:\Users\Jeff\documents\pipe tests>

At this point I don't even know if it is my messed up development environment.

Could someone please try that first pipe to see if it comes out clean?

Or, any help in pointing me to where the bug is would be most welcome.

Thank you.

Jeff Hennick


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes Problem: LITERAL & CONS

Jeff Hennick-3

Thank you.

Wow.  The direct command line works for me, too.  With either CONS or CONSOLE.  But copy/paste it into a ct.njp file and it blows up.

Well, the command line version is giving me double outputs.  That is two "Pipes is good!" and two "done". With a single literal and console, I get a single output.

On 4/9/2020 5:08 PM, ColinK wrote:
HI Jeff

If I change cons to console it works fine for me.

Z:\NetRexx-3.08-GA\bin>pipe "(ct ) literal Pipes is good! | console ? literal done | console ?"
Pipes is good!
done

Z:\NetRexx-3.08-GA\bin>pipc ctnjp
pipe (ctnjp ) literal Pipes is good! | console ? literal done | console ?

Z:\NetRexx-3.08-GA\bin>java ctnjp
Pipes is good!
done

Hope this helps

Colin

On 2020-04-09 12:47, Jeff Hennick wrote:

I have set up alias classes for CONS, all the way out to CONSOLE.  And TERM to TERMINAL.  These work just fine individually.  Then I tried a compound test and got NULL-errors.

It is a problem that is wrapping me up in knots when I try to use two or more of them in a pipe with two or more LITERAL stages.

Here is when it goes bad, with pipe (ct ) literal Pipes is good! | cons ? literal done | cons ?, which I have in the file ct.njp .  You can see both literals and cons work and the data is displayed.  Then NULL pops up.

PS C:\Users\Jeff\documents\pipe tests> pipc ct pipe (ct ) literal Pipes is good! | cons ? literal done | cons ? PS C:\Users\Jeff\documents\pipe tests> java ct done Pipes is good! java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:88) at org.netrexx.njpipes.stages.cons.run(cons.java:130) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:76) at java.lang.Thread.run(Unknown Source) java.lang.NullPointerException Exiting due to uncaught Exception null at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:88) at org.netrexx.njpipes.stages.cons.run(cons.java:130) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:76) at java.lang.Thread.run(Unknown Source) Exception in thread "Thread-4" Exiting due to uncaught Exception null java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:93) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:108) at java.lang.Thread.run(Unknown Source) Exception in thread "Thread-2" java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:93) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:108) at java.lang.Thread.run(Unknown Source) PS C:\Users\Jeff\documents\pipe tests>

But by simply removing the second LITERAL stage (and changing the endpipe to endstage), it works:

PS C:\Users\Jeff\documents\pipe tests> pipc ct pipe (ct ) literal Pipes is good! | cons | cons ? PS C:\Users\Jeff\documents\pipe tests> java ct Pipes is good! Pipes is good! PS C:\Users\Jeff\documents\pipe tests>

At this point I don't even know if it is my messed up development environment.

Could someone please try that first pipe to see if it comes out clean?

Or, any help in pointing me to where the bug is would be most welcome.

Thank you.

Jeff Hennick


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes Problem: LITERAL & CONS

rvjansen
It seems you have found an interesting problem. I have it flagged to look into tonight.
Probably, it will be part of a greater effort to make sure command line pipe definitions can go into the pipe compiler unchanged.

On 9 Apr 2020, at 23:33, Jeff Hennick <[hidden email]> wrote:

Thank you.

Wow.  The direct command line works for me, too.  With either CONS or CONSOLE.  But copy/paste it into a ct.njp file and it blows up.

Well, the command line version is giving me double outputs.  That is two "Pipes is good!" and two "done". With a single literal and console, I get a single output.

On 4/9/2020 5:08 PM, ColinK wrote:
HI Jeff

If I change cons to console it works fine for me.

Z:\NetRexx-3.08-GA\bin>pipe "(ct ) literal Pipes is good! | console ? literal done | console ?"
Pipes is good!
done

Z:\NetRexx-3.08-GA\bin>pipc ctnjp
pipe (ctnjp ) literal Pipes is good! | console ? literal done | console ?

Z:\NetRexx-3.08-GA\bin>java ctnjp
Pipes is good!
done

Hope this helps

Colin

On 2020-04-09 12:47, Jeff Hennick wrote:

I have set up alias classes for CONS, all the way out to CONSOLE.  And TERM to TERMINAL.  These work just fine individually.  Then I tried a compound test and got NULL-errors.

It is a problem that is wrapping me up in knots when I try to use two or more of them in a pipe with two or more LITERAL stages.

Here is when it goes bad, with pipe (ct ) literal Pipes is good! | cons ? literal done | cons ?, which I have in the file ct.njp .  You can see both literals and cons work and the data is displayed.  Then NULL pops up.

PS C:\Users\Jeff\documents\pipe tests> pipc ct pipe (ct ) literal Pipes is good! | cons ? literal done | cons ? PS C:\Users\Jeff\documents\pipe tests> java ct done Pipes is good! java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:88) at org.netrexx.njpipes.stages.cons.run(cons.java:130) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:76) at java.lang.Thread.run(Unknown Source) java.lang.NullPointerException Exiting due to uncaught Exception null at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:88) at org.netrexx.njpipes.stages.cons.run(cons.java:130) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:76) at java.lang.Thread.run(Unknown Source) Exception in thread "Thread-4" Exiting due to uncaught Exception null java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:93) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:108) at java.lang.Thread.run(Unknown Source) Exception in thread "Thread-2" java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:93) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:108) at java.lang.Thread.run(Unknown Source) PS C:\Users\Jeff\documents\pipe tests>

But by simply removing the second LITERAL stage (and changing the endpipe to endstage), it works:

PS C:\Users\Jeff\documents\pipe tests> pipc ct pipe (ct ) literal Pipes is good! | cons | cons ? PS C:\Users\Jeff\documents\pipe tests> java ct Pipes is good! Pipes is good! PS C:\Users\Jeff\documents\pipe tests>

At this point I don't even know if it is my messed up development environment.

Could someone please try that first pipe to see if it comes out clean?

Or, any help in pointing me to where the bug is would be most welcome.

Thank you.

Jeff Hennick


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes Problem: LITERAL & CONS

rvjansen
I can confirm the bug. It also occurs on the command line if I up-arrow fast enough. It is unpredictable, in the sense that it will go wrong mostly at the start of a sequence of tests, then it will be ok for a while, then it will miss the duplicate lines sometimes, and then it will start throwing the null pointers.

This is typical for thread-safety problems, which are unfortunately rather hard to debug. I have had cases where I never found the culprit, but got it solved by using “protect” on everything that I suspected had to do with it - heap data mostly.

I’ll see if I can spot a pattern and diagnose it, or that I can voodoo my way out of it.

btw I corrected abbreviate and its smaller siblings - have a look at the diff - I think something was left in there by accident.

Thanks for finding this by the way. NetRexx Pipes uses multithreading quite a bit - this is why we get excellent performance on multi-cpu machines where elapsed time is often a fraction of CPU time. This is the price to pay, things like these must happen sooner or later. 

One last thought: I we are very lucky, we can get away with it by properly protecting all collection classes. Notice that the null pointer is in a Linked List iterator.

René.

On 10 Apr 2020, at 09:38, René Jansen <[hidden email]> wrote:

It seems you have found an interesting problem. I have it flagged to look into tonight.
Probably, it will be part of a greater effort to make sure command line pipe definitions can go into the pipe compiler unchanged.

On 9 Apr 2020, at 23:33, Jeff Hennick <[hidden email]> wrote:

Thank you.

Wow.  The direct command line works for me, too.  With either CONS or CONSOLE.  But copy/paste it into a ct.njp file and it blows up.

Well, the command line version is giving me double outputs.  That is two "Pipes is good!" and two "done". With a single literal and console, I get a single output.

On 4/9/2020 5:08 PM, ColinK wrote:
HI Jeff

If I change cons to console it works fine for me.

Z:\NetRexx-3.08-GA\bin>pipe "(ct ) literal Pipes is good! | console ? literal done | console ?"
Pipes is good!
done

Z:\NetRexx-3.08-GA\bin>pipc ctnjp
pipe (ctnjp ) literal Pipes is good! | console ? literal done | console ?

Z:\NetRexx-3.08-GA\bin>java ctnjp
Pipes is good!
done

Hope this helps

Colin

On 2020-04-09 12:47, Jeff Hennick wrote:

I have set up alias classes for CONS, all the way out to CONSOLE.  And TERM to TERMINAL.  These work just fine individually.  Then I tried a compound test and got NULL-errors.

It is a problem that is wrapping me up in knots when I try to use two or more of them in a pipe with two or more LITERAL stages.

Here is when it goes bad, with pipe (ct ) literal Pipes is good! | cons ? literal done | cons ?, which I have in the file ct.njp .  You can see both literals and cons work and the data is displayed.  Then NULL pops up.

PS C:\Users\Jeff\documents\pipe tests> pipc ct pipe (ct ) literal Pipes is good! | cons ? literal done | cons ? PS C:\Users\Jeff\documents\pipe tests> java ct done Pipes is good! java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:88) at org.netrexx.njpipes.stages.cons.run(cons.java:130) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:76) at java.lang.Thread.run(Unknown Source) java.lang.NullPointerException Exiting due to uncaught Exception null at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:88) at org.netrexx.njpipes.stages.cons.run(cons.java:130) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:76) at java.lang.Thread.run(Unknown Source) Exception in thread "Thread-4" Exiting due to uncaught Exception null java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:93) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:108) at java.lang.Thread.run(Unknown Source) Exception in thread "Thread-2" java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:93) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:108) at java.lang.Thread.run(Unknown Source) PS C:\Users\Jeff\documents\pipe tests>

But by simply removing the second LITERAL stage (and changing the endpipe to endstage), it works:

PS C:\Users\Jeff\documents\pipe tests> pipc ct pipe (ct ) literal Pipes is good! | cons | cons ? PS C:\Users\Jeff\documents\pipe tests> java ct Pipes is good! Pipes is good! PS C:\Users\Jeff\documents\pipe tests>

At this point I don't even know if it is my messed up development environment.

Could someone please try that first pipe to see if it comes out clean?

Or, any help in pointing me to where the bug is would be most welcome.

Thank you.

Jeff Hennick


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes Problem: LITERAL & CONS

rvjansen
Let me try something. I just hit me (from reading the stack trace with some more attention) that I probably know who did this to us -it was me, or rather, the people who did not contribute money for the new Java implementation for OS/2. I put in support for multiple output streams on SAY (netrexx.lang.Rexx.IO.say) in 3.07, and put a nice ConcurrentLinkedDeque to keep the multiple outputstreams in.

But when it turned out that this collection class was too new for the most modern version of Java on OS/2, I fell back to a linked list, knowing this was slightly unsafe. You might have found why.

I’ll keep you posted.

René.

On 10 Apr 2020, at 23:18, René Jansen <[hidden email]> wrote:

I can confirm the bug. It also occurs on the command line if I up-arrow fast enough. It is unpredictable, in the sense that it will go wrong mostly at the start of a sequence of tests, then it will be ok for a while, then it will miss the duplicate lines sometimes, and then it will start throwing the null pointers.

This is typical for thread-safety problems, which are unfortunately rather hard to debug. I have had cases where I never found the culprit, but got it solved by using “protect” on everything that I suspected had to do with it - heap data mostly.

I’ll see if I can spot a pattern and diagnose it, or that I can voodoo my way out of it.

btw I corrected abbreviate and its smaller siblings - have a look at the diff - I think something was left in there by accident.

Thanks for finding this by the way. NetRexx Pipes uses multithreading quite a bit - this is why we get excellent performance on multi-cpu machines where elapsed time is often a fraction of CPU time. This is the price to pay, things like these must happen sooner or later. 

One last thought: I we are very lucky, we can get away with it by properly protecting all collection classes. Notice that the null pointer is in a Linked List iterator.

René.

On 10 Apr 2020, at 09:38, René Jansen <[hidden email]> wrote:

It seems you have found an interesting problem. I have it flagged to look into tonight.
Probably, it will be part of a greater effort to make sure command line pipe definitions can go into the pipe compiler unchanged.

On 9 Apr 2020, at 23:33, Jeff Hennick <[hidden email]> wrote:

Thank you.

Wow.  The direct command line works for me, too.  With either CONS or CONSOLE.  But copy/paste it into a ct.njp file and it blows up.

Well, the command line version is giving me double outputs.  That is two "Pipes is good!" and two "done". With a single literal and console, I get a single output.

On 4/9/2020 5:08 PM, ColinK wrote:
HI Jeff

If I change cons to console it works fine for me.

Z:\NetRexx-3.08-GA\bin>pipe "(ct ) literal Pipes is good! | console ? literal done | console ?"
Pipes is good!
done

Z:\NetRexx-3.08-GA\bin>pipc ctnjp
pipe (ctnjp ) literal Pipes is good! | console ? literal done | console ?

Z:\NetRexx-3.08-GA\bin>java ctnjp
Pipes is good!
done

Hope this helps

Colin

On 2020-04-09 12:47, Jeff Hennick wrote:

I have set up alias classes for CONS, all the way out to CONSOLE.  And TERM to TERMINAL.  These work just fine individually.  Then I tried a compound test and got NULL-errors.

It is a problem that is wrapping me up in knots when I try to use two or more of them in a pipe with two or more LITERAL stages.

Here is when it goes bad, with pipe (ct ) literal Pipes is good! | cons ? literal done | cons ?, which I have in the file ct.njp .  You can see both literals and cons work and the data is displayed.  Then NULL pops up.

PS C:\Users\Jeff\documents\pipe tests> pipc ct pipe (ct ) literal Pipes is good! | cons ? literal done | cons ? PS C:\Users\Jeff\documents\pipe tests> java ct done Pipes is good! java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:88) at org.netrexx.njpipes.stages.cons.run(cons.java:130) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:76) at java.lang.Thread.run(Unknown Source) java.lang.NullPointerException Exiting due to uncaught Exception null at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:88) at org.netrexx.njpipes.stages.cons.run(cons.java:130) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:76) at java.lang.Thread.run(Unknown Source) Exception in thread "Thread-4" Exiting due to uncaught Exception null java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:93) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:108) at java.lang.Thread.run(Unknown Source) Exception in thread "Thread-2" java.lang.NullPointerException at java.util.LinkedList$ListItr.next(Unknown Source) at netrexx.lang.RexxIO.Say(RexxIO.java:121) at netrexx.lang.RexxIO.Say(RexxIO.java:93) at org.netrexx.njpipes.pipes.ThreadPool.run(ThreadPool.java:108) at java.lang.Thread.run(Unknown Source) PS C:\Users\Jeff\documents\pipe tests>

But by simply removing the second LITERAL stage (and changing the endpipe to endstage), it works:

PS C:\Users\Jeff\documents\pipe tests> pipc ct pipe (ct ) literal Pipes is good! | cons | cons ? PS C:\Users\Jeff\documents\pipe tests> java ct Pipes is good! Pipes is good! PS C:\Users\Jeff\documents\pipe tests>

At this point I don't even know if it is my messed up development environment.

Could someone please try that first pipe to see if it comes out clean?

Or, any help in pointing me to where the bug is would be most welcome.

Thank you.

Jeff Hennick


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/




_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes Problem: LITERAL & CONS

rvjansen
I have pushed the first part of the fix. I cannot get it to null pointer anymore. There is still a problem of INCORROUT. The pipe as I tried it

pipe "literal Pipes is good! | console | literal wow | console "

should give 2 Pipes is good! and one wow - in officially undefined sequence I think.

It does not on my system, I get 4 Pipes is good! and 2 wows. 

Also, your example:

➜  netrexx-code git:(master) pipe "literal Pipes is good! | cons ? literal done | cons ?"
done
done
Pipes is good!
Pipes is good! 

So that is some duplication that should not be there. I’ll sleep over this and look into it in the morning.

Can you explain what the pipe ends and the stage ends are supposed to do?

best regards,

René.

On 10 Apr 2020, at 23:29, René Jansen <[hidden email]> wrote:

Let me try something. I just hit me (from reading the stack trace with some more attention) that I probably know who did this to us -it was me, or rather, the people who did not contribute money for the new Java implementation for OS/2. I put in support for multiple output streams on SAY (netrexx.lang.Rexx.IO.say) in 3.07, and put a nice ConcurrentLinkedDeque to keep the multiple outputstreams in.

But when it turned out that this collection class was too new for the most modern version of Java on OS/2, I fell back to a linked list, knowing this was slightly unsafe. You might have found why.

I’ll keep you posted.

René.

On 10 Apr 2020, at 23:18, René Jansen <[hidden email]> wrote:

I can confirm the bug. It also occurs on the command line if I up-arrow fast enough. It is unpredictable, in the sense that it will go wrong mostly at the start of a sequence of tests, then it will be ok for a while, then it will miss the duplicate lines sometimes, and then it will start throwing the null pointers.



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes Problem: LITERAL & CONS

rvjansen
Fixed it. The issue here is that RexxIO.Say (in its various forms) are static methods - always were. Instead of trying to build that into instance methods, and forcing a rebuild of all NetRexx code that is out there since 1996, I thought it was better to tackle it in the console stage. This now has a RexxIO.pipeSay() method call, which can be called polymorphically just like say, but unlike Say, it is just a method call without special treatment by the compiler.

It seems that there was not much ’say’ing going on from multiple threads, and I wonder what the semantics should be. A process has one stdout, so I wonder what happens if we fire at that from multiple threads - do we get character overlaps, or locking issues. In any case, this now is probably fixed, including giving the output lines in random sequence, just like on CMS - which I gather is not multithreaded, but dispatched in a first come first served way.

Thanks again for noticing this before NetRexx 3.09 was released. The fix is checked in and ready for testing. Please let me know.

René. 

On 11 Apr 2020, at 00:04, René Jansen <[hidden email]> wrote:

I have pushed the first part of the fix. I cannot get it to null pointer anymore. There is still a problem of INCORROUT. The pipe as I tried it

pipe "literal Pipes is good! | console | literal wow | console "

should give 2 Pipes is good! and one wow - in officially undefined sequence I think.

It does not on my system, I get 4 Pipes is good! and 2 wows. 

Also, your example:

➜  netrexx-code git:(master) pipe "literal Pipes is good! | cons ? literal done | cons ?"
done
done
Pipes is good!
Pipes is good! 

So that is some duplication that should not be there. I’ll sleep over this and look into it in the morning.

Can you explain what the pipe ends and the stage ends are supposed to do?

best regards,

René.

On 10 Apr 2020, at 23:29, René Jansen <[hidden email]> wrote:

Let me try something. I just hit me (from reading the stack trace with some more attention) that I probably know who did this to us -it was me, or rather, the people who did not contribute money for the new Java implementation for OS/2. I put in support for multiple output streams on SAY (netrexx.lang.Rexx.IO.say) in 3.07, and put a nice ConcurrentLinkedDeque to keep the multiple outputstreams in.

But when it turned out that this collection class was too new for the most modern version of Java on OS/2, I fell back to a linked list, knowing this was slightly unsafe. You might have found why.

I’ll keep you posted.

René.

On 10 Apr 2020, at 23:18, René Jansen <[hidden email]> wrote:

I can confirm the bug. It also occurs on the command line if I up-arrow fast enough. It is unpredictable, in the sense that it will go wrong mostly at the start of a sequence of tests, then it will be ok for a while, then it will miss the duplicate lines sometimes, and then it will start throwing the null pointers.




_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes Problem: LITERAL & CONS

Jeff Hennick-3

Great!  Thanks, from all of us.

pipe "literal Pipes is good! | cons ? literal done | cons ?"
Can you explain what the pipe ends and the stage ends are supposed to do?
My intention was to make two independent pipelines.  The first should not be passed to the second literal.  The output should be, I think, one copy of each literal.  It started out like this to test all the aliases

pipe "literal Pipes is good! | cons | conso |console | term | ... terminal ? literal done | cons ?"

It worked until I added the done stuff.

On 4/11/2020 9:04 AM, René Jansen wrote:
Fixed it. The issue here is that RexxIO.Say (in its various forms) are static methods - always were. Instead of trying to build that into instance methods, and forcing a rebuild of all NetRexx code that is out there since 1996, I thought it was better to tackle it in the console stage. This now has a RexxIO.pipeSay() method call, which can be called polymorphically just like say, but unlike Say, it is just a method call without special treatment by the compiler.

It seems that there was not much ’say’ing going on from multiple threads, and I wonder what the semantics should be. A process has one stdout, so I wonder what happens if we fire at that from multiple threads - do we get character overlaps, or locking issues. In any case, this now is probably fixed, including giving the output lines in random sequence, just like on CMS - which I gather is not multithreaded, but dispatched in a first come first served way.

Thanks again for noticing this before NetRexx 3.09 was released. The fix is checked in and ready for testing. Please let me know.

René. 

On 11 Apr 2020, at 00:04, René Jansen <[hidden email]> wrote:

I have pushed the first part of the fix. I cannot get it to null pointer anymore. There is still a problem of INCORROUT. The pipe as I tried it

pipe "literal Pipes is good! | console | literal wow | console "

should give 2 Pipes is good! and one wow - in officially undefined sequence I think.

It does not on my system, I get 4 Pipes is good! and 2 wows. 

Also, your example:

➜  netrexx-code git:(master) pipe "literal Pipes is good! | cons ? literal done | cons ?"
done
done
Pipes is good!
Pipes is good! 

So that is some duplication that should not be there. I’ll sleep over this and look into it in the morning.

Can you explain what the pipe ends and the stage ends are supposed to do?

best regards,

René.

On 10 Apr 2020, at 23:29, René Jansen <[hidden email]> wrote:

Let me try something. I just hit me (from reading the stack trace with some more attention) that I probably know who did this to us -it was me, or rather, the people who did not contribute money for the new Java implementation for OS/2. I put in support for multiple output streams on SAY (netrexx.lang.Rexx.IO.say) in 3.07, and put a nice ConcurrentLinkedDeque to keep the multiple outputstreams in.

But when it turned out that this collection class was too new for the most modern version of Java on OS/2, I fell back to a linked list, knowing this was slightly unsafe. You might have found why.

I’ll keep you posted.

René.

On 10 Apr 2020, at 23:18, René Jansen <[hidden email]> wrote:

I can confirm the bug. It also occurs on the command line if I up-arrow fast enough. It is unpredictable, in the sense that it will go wrong mostly at the start of a sequence of tests, then it will be ok for a while, then it will miss the duplicate lines sometimes, and then it will start throwing the null pointers.




_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes Problem: LITERAL & CONS

rvjansen
I googled a bit on this, and generally when doing console I/O from mulitiple threads, you end up with a jumbled, unpredictable mess. There is a tool, gnu parallel, that seems to alleviate this a bit, but I did not try it yet.

Jumbled mess or not, we have to make sure that only one copy of each ’say’s output ends up in there, so the multithread bug in ’say’ (which now is sidestepped for the CONSOLE stage) needs to be fixed still.

Speaking about bugs, we do not have a bug tracker currently. I am not using the one from SourceForge, because the latter is still on probation. Our previous bug tracker was left at Kenai and I never had the time to reload the bug in there to a new tracker. I always thought that the bug reports need to part of the repository and not of some shell around that.

I am experimenting with git-bug, which very well may fit the bill. I will let the list know in a few days how this works out. 

René.

On 12 Apr 2020, at 22:51, Jeff Hennick <[hidden email]> wrote:

It seems that there was not much ’say’ing going on from multiple threads, and I wonder what the semantics should be. A process has one stdout, so I wonder what happens if we fire at that from multiple threads - do we get character overlaps, or locking issues. In any case, this now is probably fixed, including giving the output lines in random sequence, just like on CMS - which I gather is not multithreaded, but dispatched in a first come first served way.



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: Pipes Problem: LITERAL & CONS

ColinK
In reply to this post by rvjansen
Excellent Rene!  Thanks for figuring it out.  

Interestingly, by per chance, I rebooted my PC and now my pipe work is getting those null exceptions whereas it worked just fine before. 

Now I just need to read up on git to pull down the changed bits that fixes this. :-)

The examples you gave in your earlier note work as expected for me.

Z:\NetRexx-3.08-GA\bin>pipe "literal Pipes is good! | console | literal wow | console "
Pipes is good!
wow
Pipes is good!

Z:\NetRexx-3.08-GA\bin> pipe "literal Pipes is good! | console ? literal done | console ?"
Pipes is good!
done

Z:\NetRexx-3.08-GA\bin> pipe "literal Pipes is good! | console ? literal done | console ?"
Pipes is good!
done

Z:\NetRexx-3.08-GA\bin>java ctnjp
Pipes is good!
done

Z:\NetRexx-3.08-GA\bin>java ctnjp
done
Pipes is good!

done sometimes shows up prior to Pipes is good! - but I would expect that in a multithreaded environment.

Cheers
Colin

On 2020-04-11 06:04, René Jansen wrote:
Fixed it. The issue here is that RexxIO.Say (in its various forms) are static methods - always were. Instead of trying to build that into instance methods, and forcing a rebuild of all NetRexx code that is out there since 1996, I thought it was better to tackle it in the console stage. This now has a RexxIO.pipeSay() method call, which can be called polymorphically just like say, but unlike Say, it is just a method call without special treatment by the compiler.

It seems that there was not much ’say’ing going on from multiple threads, and I wonder what the semantics should be. A process has one stdout, so I wonder what happens if we fire at that from multiple threads - do we get character overlaps, or locking issues. In any case, this now is probably fixed, including giving the output lines in random sequence, just like on CMS - which I gather is not multithreaded, but dispatched in a first come first served way.

Thanks again for noticing this before NetRexx 3.09 was released. The fix is checked in and ready for testing. Please let me know.

René. 

On 11 Apr 2020, at 00:04, René Jansen <[hidden email]> wrote:

I have pushed the first part of the fix. I cannot get it to null pointer anymore. There is still a problem of INCORROUT. The pipe as I tried it

pipe "literal Pipes is good! | console | literal wow | console "

should give 2 Pipes is good! and one wow - in officially undefined sequence I think.

It does not on my system, I get 4 Pipes is good! and 2 wows. 

Also, your example:

➜  netrexx-code git:(master) pipe "literal Pipes is good! | cons ? literal done | cons ?"
done
done
Pipes is good!
Pipes is good! 

So that is some duplication that should not be there. I’ll sleep over this and look into it in the morning.

Can you explain what the pipe ends and the stage ends are supposed to do?

best regards,

René.

On 10 Apr 2020, at 23:29, René Jansen <[hidden email]> wrote:

Let me try something. I just hit me (from reading the stack trace with some more attention) that I probably know who did this to us -it was me, or rather, the people who did not contribute money for the new Java implementation for OS/2. I put in support for multiple output streams on SAY (netrexx.lang.Rexx.IO.say) in 3.07, and put a nice ConcurrentLinkedDeque to keep the multiple outputstreams in.

But when it turned out that this collection class was too new for the most modern version of Java on OS/2, I fell back to a linked list, knowing this was slightly unsafe. You might have found why.

I’ll keep you posted.

René.

On 10 Apr 2020, at 23:18, René Jansen <[hidden email]> wrote:

I can confirm the bug. It also occurs on the command line if I up-arrow fast enough. It is unpredictable, in the sense that it will go wrong mostly at the start of a sequence of tests, then it will be ok for a while, then it will miss the duplicate lines sometimes, and then it will start throwing the null pointers.




_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: [EXTERNAL] Re: Pipes Problem: LITERAL & CONS

Nix, Robert P.

In the first example, don’t put the first | console in: literal produces its record, and then passes anything in the pipe prior to the stage.

 

So:

 

pipe “literal Pipes is good! | literal wow | console”

 

Should print your two records each one time.

 

--

Robert P. Nix | Sr IT Systems Engineer | Application Hosting

Mayo Clinic | 200 First Street SW | Rochester, MN 55905

507-284-0844 | [hidden email]

ДУМАТЬ

 

 

From: <[hidden email]> on behalf of ColinK <[hidden email]>
Reply-To: IBM Netrexx <[hidden email]>
Date: Monday, April 13, 2020 at 4:06 PM
To: IBM Netrexx <[hidden email]>
Subject: [EXTERNAL] Re: [Ibm-netrexx] Pipes Problem: LITERAL & CONS

 

Excellent Rene!  Thanks for figuring it out.  

Interestingly, by per chance, I rebooted my PC and now my pipe work is getting those null exceptions whereas it worked just fine before. 

Now I just need to read up on git to pull down the changed bits that fixes this. :-)

The examples you gave in your earlier note work as expected for me.

Z:\NetRexx-3.08-GA\bin>pipe "literal Pipes is good! | console | literal wow | console "
Pipes is good!
wow
Pipes is good!

Z:\NetRexx-3.08-GA\bin> pipe "literal Pipes is good! | console ? literal done | console ?"
Pipes is good!
done

Z:\NetRexx-3.08-GA\bin> pipe "literal Pipes is good! | console ? literal done | console ?"
Pipes is good!
done

Z:\NetRexx-3.08-GA\bin>java ctnjp
Pipes is good!
done

Z:\NetRexx-3.08-GA\bin>java ctnjp
done
Pipes is good!

done sometimes shows up prior to Pipes is good! - but I would expect that in a multithreaded environment.

Cheers
Colin

On 2020-04-11 06:04, René Jansen wrote:

Fixed it. The issue here is that RexxIO.Say (in its various forms) are static methods - always were. Instead of trying to build that into instance methods, and forcing a rebuild of all NetRexx code that is out there since 1996, I thought it was better to tackle it in the console stage. This now has a RexxIO.pipeSay() method call, which can be called polymorphically just like say, but unlike Say, it is just a method call without special treatment by the compiler.

 

It seems that there was not much ’say’ing going on from multiple threads, and I wonder what the semantics should be. A process has one stdout, so I wonder what happens if we fire at that from multiple threads - do we get character overlaps, or locking issues. In any case, this now is probably fixed, including giving the output lines in random sequence, just like on CMS - which I gather is not multithreaded, but dispatched in a first come first served way.

 

Thanks again for noticing this before NetRexx 3.09 was released. The fix is checked in and ready for testing. Please let me know.

 

René. 



On 11 Apr 2020, at 00:04, René Jansen <[hidden email]> wrote:

 

I have pushed the first part of the fix. I cannot get it to null pointer anymore. There is still a problem of INCORROUT. The pipe as I tried it

 

pipe "literal Pipes is good! | console | literal wow | console "

 

should give 2 Pipes is good! and one wow - in officially undefined sequence I think.

 

It does not on my system, I get 4 Pipes is good! and 2 wows. 

 

Also, your example:

 

 netrexx-code git:(master) pipe "literal Pipes is good! | cons ? literal done | cons ?"

done

done

Pipes is good!

Pipes is good! 

 

So that is some duplication that should not be there. I’ll sleep over this and look into it in the morning.

 

Can you explain what the pipe ends and the stage ends are supposed to do?

 

best regards,

 

René.



On 10 Apr 2020, at 23:29, René Jansen <[hidden email]> wrote:

 

Let me try something. I just hit me (from reading the stack trace with some more attention) that I probably know who did this to us -it was me, or rather, the people who did not contribute money for the new Java implementation for OS/2. I put in support for multiple output streams on SAY (netrexx.lang.Rexx.IO.say) in 3.07, and put a nice ConcurrentLinkedDeque to keep the multiple outputstreams in.

 

But when it turned out that this collection class was too new for the most modern version of Java on OS/2, I fell back to a linked list, knowing this was slightly unsafe. You might have found why.

 

I’ll keep you posted.

 

René.



On 10 Apr 2020, at 23:18, René Jansen <[hidden email]> wrote:

 

I can confirm the bug. It also occurs on the command line if I up-arrow fast enough. It is unpredictable, in the sense that it will go wrong mostly at the start of a sequence of tests, then it will be ok for a while, then it will miss the duplicate lines sometimes, and then it will start throwing the null pointers.

 

 

 



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/
 




_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

Reply | Threaded
Open this post in threaded view
|

Re: [EXTERNAL] Re: Pipes Problem: LITERAL & CONS

Jeff Hennick-3

Robert,

You are exactly right in your analysis.

I am in the long process of producing a suite of full tests for all the stages provided with njPipes -- and fixing the bugs that have slipped in.  My original problem was testing the various abbreviations for LITERAL and its alias TERMINAL.  So I had a long string of these stages, followed by a second pipe of the single "done."  This showed the bug, and it was in shortening it as a reproducible example that was in the post:

   pipe "literal Pipes is good! | console ? literal wow | console"

That example had a pipe-end, not just a stage-separator after the first CONSOLE.  That configuration, too, should have printed just the two records, but it was not.  René has now fixed for the next release.

For the record, CMS pipelines would handle both configurations the same, but njPipes might not as it is multi-threaded and we do not know which of the two pipelines will finish first and display.  The two records can come out in any order.  I see this in writing complex pipes testing different options.  To avoid it, instead of multiple TERMINAL stages, I prepend them with the order I want then route them all through this:


 o: faninany |
    sort |
    specs 2-* 1 |
    cons ?
 

Jeff Hennick

On 4/20/2020 5:14 PM, Nix, Robert P. wrote:

In the first example, don’t put the first | console in: literal produces its record, and then passes anything in the pipe prior to the stage.

 

So:

 

pipe “literal Pipes is good! | literal wow | console”

 

Should print your two records each one time.

 

--

Robert P. Nix | Sr IT Systems Engineer | Application Hosting

Mayo Clinic | 200 First Street SW | Rochester, MN 55905

507-284-0844 | [hidden email]

ДУМАТЬ

 

 

From: [hidden email] on behalf of ColinK [hidden email]
Reply-To: IBM Netrexx [hidden email]
Date: Monday, April 13, 2020 at 4:06 PM
To: IBM Netrexx [hidden email]
Subject: [EXTERNAL] Re: [Ibm-netrexx] Pipes Problem: LITERAL & CONS

 

Excellent Rene!  Thanks for figuring it out.  

Interestingly, by per chance, I rebooted my PC and now my pipe work is getting those null exceptions whereas it worked just fine before. 

Now I just need to read up on git to pull down the changed bits that fixes this. :-)

The examples you gave in your earlier note work as expected for me.

Z:\NetRexx-3.08-GA\bin>pipe "literal Pipes is good! | console | literal wow | console "
Pipes is good!
wow
Pipes is good!

Z:\NetRexx-3.08-GA\bin> pipe "literal Pipes is good! | console ? literal done | console ?"
Pipes is good!
done

Z:\NetRexx-3.08-GA\bin> pipe "literal Pipes is good! | console ? literal done | console ?"
Pipes is good!
done

Z:\NetRexx-3.08-GA\bin>java ctnjp
Pipes is good!
done

Z:\NetRexx-3.08-GA\bin>java ctnjp
done
Pipes is good!

done sometimes shows up prior to Pipes is good! - but I would expect that in a multithreaded environment.

Cheers
Colin

On 2020-04-11 06:04, René Jansen wrote:

Fixed it. The issue here is that RexxIO.Say (in its various forms) are static methods - always were. Instead of trying to build that into instance methods, and forcing a rebuild of all NetRexx code that is out there since 1996, I thought it was better to tackle it in the console stage. This now has a RexxIO.pipeSay() method call, which can be called polymorphically just like say, but unlike Say, it is just a method call without special treatment by the compiler.

 

It seems that there was not much ’say’ing going on from multiple threads, and I wonder what the semantics should be. A process has one stdout, so I wonder what happens if we fire at that from multiple threads - do we get character overlaps, or locking issues. In any case, this now is probably fixed, including giving the output lines in random sequence, just like on CMS - which I gather is not multithreaded, but dispatched in a first come first served way.

 

Thanks again for noticing this before NetRexx 3.09 was released. The fix is checked in and ready for testing. Please let me know.

 

René. 



On 11 Apr 2020, at 00:04, René Jansen <[hidden email]> wrote:

 

I have pushed the first part of the fix. I cannot get it to null pointer anymore. There is still a problem of INCORROUT. The pipe as I tried it

 

pipe "literal Pipes is good! | console | literal wow | console "

 

should give 2 Pipes is good! and one wow - in officially undefined sequence I think.

 

It does not on my system, I get 4 Pipes is good! and 2 wows. 

 

Also, your example:

 

 netrexx-code git:(master) pipe "literal Pipes is good! | cons ? literal done | cons ?"

done

done

Pipes is good!

Pipes is good! 

 

So that is some duplication that should not be there. I’ll sleep over this and look into it in the morning.

 

Can you explain what the pipe ends and the stage ends are supposed to do?

 

best regards,

 

René.



On 10 Apr 2020, at 23:29, René Jansen <[hidden email]> wrote:

 

Let me try something. I just hit me (from reading the stack trace with some more attention) that I probably know who did this to us -it was me, or rather, the people who did not contribute money for the new Java implementation for OS/2. I put in support for multiple output streams on SAY (netrexx.lang.Rexx.IO.say) in 3.07, and put a nice ConcurrentLinkedDeque to keep the multiple outputstreams in.

 

But when it turned out that this collection class was too new for the most modern version of Java on OS/2, I fell back to a linked list, knowing this was slightly unsafe. You might have found why.

 

I’ll keep you posted.

 

René.



On 10 Apr 2020, at 23:18, René Jansen <[hidden email]> wrote:

 

I can confirm the bug. It also occurs on the command line if I up-arrow fast enough. It is unpredictable, in the sense that it will go wrong mostly at the start of a sequence of tests, then it will be ok for a while, then it will miss the duplicate lines sometimes, and then it will start throwing the null pointers.

 

 

 



_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/
 




_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/


_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/