Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

If you get a "unable to load libjvm.so" error you should reinstall java, or try a different Java package.

If you get an error about missing javaembedaccompanied by "Warning:  /usr/bin/ld: cannot find -ljavaembed", the HPCC install does not include the plugin feature and needs to be reinstalled from a distro with the plugin feature.

...

sudo service hpcc-init restart  => command for restarting

 

Call your Java from ECL

Currently, you can only pass primitive data types to and from a Java plugin (String, long, etc.)

...

      STRING segment() := IMPORT(java,'org/hpccsystems/Segmenter.SegmentText:(Ljava/lang/String;Ljava/lang/String;)Ljava/lang/String;');

You must include the entire path to the class being called within the jar to the IMPORT statement.

To pass complex data from a Java class, the best practice is to pass it as an xml or otherwise encoded/delimited string, and then break the xml/delimited fields into separate fields for processing. Check the simple Java Integration sample below for an example of how to turn an xml string returned from a java class into a dataset. An additional example can be found at https://github.com/hpcc-systems/TextAnalytics/blob/master/hpcc/ecl/medex/BWR_MedexDrugSignatures.ecl

Troubleshooting

  1. You can run the sample java, but get a class not found error when you try to run yours.

HPCC can't find your jar or class in its classpath. First things to check:

    1. Verify that the entry in /etc/HPCCSystems/environment.conf is pointing to the correct jar file
    2. Verify that your jar file is located where expected
    3. Verify that the class package location in the jar matches the path defined in the ECL IMPORT Statement
    4. Verify that the jar is correctly deployed on all thor nodes in the cluster.
    5. Verify that the jar file is world readable and executable.
    6. Verify that the thor cluster was restarted after you added your entry to the classpath.

If all of these checks pass, look for a JVM Startup entry in the eclagent.log to verify that the JVM was loaded, and check what is actually being used as the classpath. You should see something like:

              "javaembed: Setting JVM Option: -Djava.class.path=/opt/HPCCSystems/classes:/opt/HPCCSystems/classes/myjava.jar:."

 

2. You try running your java, and get an out of memory or heapspace error.

In HPCC, the JVM is started when Thor starts and continues to run until Thor is restarted. Any java classes loaded into memory during this time remain there.

You can increase your maximum memory for the JVM by adding the following line to /etc/HPCCSystems/environment.conf:  jvmoptions=-Xmx1024m

Alternately, if your java plugin requires a large amount of data loaded in memory (such as lexicons, etc. needed by Natural Language Processing java jars), you can add a method to your java jar to clear out cached data after a process is run so that it does not continue to take up space in the Java heap. For example:

Code Block
import java;
STRING segment() := IMPORT(java,'org/hpccsystems/Segmenter.SegmentText:(Ljava/lang/String;Ljava/lang/String;)Ljava/lang/String;');
STRING clearcache() := IMPORT(java,'org/hpccsystems/Segmenter.ClearCache:(Ljava/lang/Boolean)Ljava/lang/String;');

SEQUENTIAL(output(segment('text to segment')),
           output(clearcache(true))
)

 

 

 

 

 

6. THE END

Feel free to raise an issue at http://track.hpccsystems.com if it does not work as expected. I assure you that it will be addressed promptly.

...