hadoop MapReduce

[janne rudwall-3]

Hi!

Need help on this one  (HADOOP/MAPREDUCE)

How to code this in netrexx?   OutputCollector<Text, IntWritable> output    

    map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter)

 

running java2nrx on

(eg -- sample code WordCountMapper.java)

 

 

import java.io.IOException;

import java.util.StringTokenizer;

 

import org.apache.hadoop.io.*;

import org.apache.hadoop.mapred.*;

 

public class b extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable>

{

      //hadoop supported data types

      private final static IntWritable one = new IntWritable(1);

      private Text word = new Text();

 

      //map method that performs the tokenizer job and framing the initial key value pairs

      public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException

      {

            //taking one line at a time and tokenizing the same

            String line = value.toString();

          StringTokenizer tokenizer = new StringTokenizer(line);

 

          //iterating through all the words available in that line and forming the key value pair

            while (tokenizer.hasMoreTokens())

            {

               word.set(tokenizer.nextToken());

               //sending to output collector which inturn passes the same to reducer

                 output.collect(word, one);

            }

       }

}

 

OUTPUT... ! ALL SEEMS FINE but What to do with /* **nonrx**  <LongWritable, Text, Text, IntWritable> ** */   ???????

 

import java.io.IOException

import java.util.StringTokenizer

import org.apache.hadoop.io.

import org.apache.hadoop.mapred.

 

class b public  extends MapReduceBase implements Mapper /* **nonrx**  <LongWritable, Text, Text, IntWritable> ** */

 

    properties private constant

    one = IntWritable IntWritable(1)

 

    properties private

    word = Text Text()

 

    method map( key=LongWritable,  value=Text,  output=OutputCollector /* **nonrx**  <Text, IntWritable> ** */,  reporter=Reporter )  signals IOException

        line = String

        line = value.toString()

        tokenizer = StringTokenizer

        tokenizer = StringTokenizer(line)

        loop while tokenizer.hasMoreTokens()

            word.set(tokenizer.nextToken())

            output.collect(word, one)

 

        end

 

compile gives this..

NetRexx portable processor, version NetRexx 3.01, build 40-20120823-0156

Copyright (c) RexxLA, 2011,2012.  All rights reserved.

Parts Copyright (c) IBM Corporation, 1995,2008.

Program b.nrx

  === class b ===

    method map(LongWritable,Text,OutputCollector,Reporter)

  8 +++ class b public  extends MapReduceBase implements Mapper /* **nonrx**  <LongWritable, Text, Text, IntWritable> ** */

    +++       ^

    +++ Error: Class is not abstract, yet it does not implement method 'map(Object,Object,OutputCollector,Reporter)' from abstract class 'org.apache.hadoop.mapred.Mapper'

Compilation of 'b.nrx' failed [one error]

 

_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/

[Marc Remes-2]

Janne,

we had a discussion on this a while back,

The translator attempts to implement the suggestion in
http://ibm-netrexx.215625.n3.nabble.com/Java-Type-declarations-for-NetRexx-td3669984.html#a3670042

The issue here is your method implementation does not match the inherited method
  method map(LongWritable,Text, ..
vs
  method map(Object,Object, ..

 From the Language Ref
<<
If the current class is not an interface class then it must implement (provide non-abstract methods for)
all the methods inherited from the interface classes in the implements list.
 >>

Try

    method map( _key=Object,  _value=Object,  output=OutputCollector ,  reporter=Reporter) signals IOException
         key = LongWritable _key
         value = Text _value

HTH

On 02/05/2013 07:19 PM, janne rudwall wrote:

> Hi!
> Need help on this one  (HADOOP/MAPREDUCE)
> How to code this in netrexx? *OutputCollector<Text, IntWritable> output *
> *    map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter)*
> running java2nrx on
>
> (eg -- sample code WordCountMapper.java)
> import java.io.IOException;
> import java.util.StringTokenizer;
> import org.apache.hadoop.io.*;
> import org.apache.hadoop.mapred.*;
> public class b extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable>
> {
>        //hadoop supported data types
>        private final static IntWritable one = new IntWritable(1);
>        private Text word = new Text();
>        //map method that performs the tokenizer job and framing the initial key value pairs
>        public void map(LongWritable key, Text value, *OutputCollector<Text, IntWritable> outpu*t, Reporter reporter)
> throws IOException
>        {
>              //taking one line at a time and tokenizing the same
>              String line = value.toString();
>            StringTokenizer tokenizer = new StringTokenizer(line);
>            //iterating through all the words available in that line and forming the key value pair
>              while (tokenizer.hasMoreTokens())
>              {
>                 word.set(tokenizer.nextToken());
>                 //sending to output collector which inturn passes the same to reducer
>                   output.collect(word, one);
>              }
>         }
> }
> *OUTPUT... ! ALL SEEMS FINE but What to do with /* **nonrx**  <LongWritable, Text, Text, IntWritable> ** */   ???????*
> import java.io.IOException
> import java.util.StringTokenizer
> import org.apache.hadoop.io.
> import org.apache.hadoop.mapred.
> class b public  extends MapReduceBase implements Mapper /* **nonrx**  <LongWritable, Text, Text, IntWritable> ** */
>      properties private constant
>      one = IntWritable IntWritable(1)
>      properties private
>      word = Text Text()
>      method map( key=LongWritable,  value=Text,  output=OutputCollector /* **nonrx**  <Text, IntWritable> ** */,
> reporter=Reporter )  signals IOException
>          line = String
>          line = value.toString()
>          tokenizer = StringTokenizer
>          tokenizer = StringTokenizer(line)
>          loop while tokenizer.hasMoreTokens()
>              word.set(tokenizer.nextToken())
>              output.collect(word, one)
>          end
> *compile gives this..*
> NetRexx portable processor, version NetRexx 3.01, build 40-20120823-0156
> Copyright (c) RexxLA, 2011,2012.  All rights reserved.
> Parts Copyright (c) IBM Corporation, 1995,2008.
> Program b.nrx
>    === class b ===
>      method map(LongWritable,Text,OutputCollector,Reporter)
>    8 +++ class b public  extends MapReduceBase implements Mapper /* **nonrx**  <LongWritable, Text, Text, IntWritable> ** */
>      +++       ^
>      +++ Error: Class is not abstract, yet it does not implement method 'map(Object,Object,OutputCollector,Reporter)'
> from abstract class 'org.apache.hadoop.mapred.Mapper'
> Compilation of 'b.nrx' failed [one error]
>
>
> _______________________________________________
> Ibm-netrexx mailing list
> [hidden email]
> Online Archive : http://ibm-netrexx.215625.n3.nabble.com/
>

--

Marc

_______________________________________________
Ibm-netrexx mailing list
[hidden email]
Online Archive : http://ibm-netrexx.215625.n3.nabble.com/