This project is available as a student work experience opportunity with HPCC Systems. Curious about other projects we are offering? Take a look at our Ideas List.

Student work experience opportunities also exist for students who want to suggest their own project idea. Project suggestions must be relevant to HPCC Systems and of benefit to our open source community. 

Find out about the HPCC Systems Summer Internship Program.

Project Description

There are several situations where it would be useful to reverse the order of a dataset. This could be done more efficiently than a sort.  The change would need to cover the following particular cases:

• inline datasets. (These could generally iterate in reverse, imposing no run time overhead)
• grouped datasets - reverse just within the group
• local
• global.

There is also opportunity for various optimizations - e.g., REVERSE(SORT())->SORT'

The objective of this project is to implement a REVERSE activity in HPCC Systems. The proposal for this project needs to describe the methods that would be used to perform the following:

1. reverse an incoming stream of rows that can all be held in memory
2. reverse an incoming stream of rows up to 10% of which can be held in memory 
3. reverse multiple streams of rows on an ordered set of machines, where the rows supplied on each node must be reversed, and the rows streams must also be reversed between the nodes.

For each example the rows will need to be sequentially output in reverse order.  There is no random access to rows in the incoming row stream.

If you are interested in this project, please contact Jake Smith.

Completion of this project involves:

• grouped and local reverse in hthor and roxie
• grouped reverse in thor
• local reverse in thor, including when the rows will not fit into memory
• global reverse in thor

By the mid term review we would expect you to have:

• Produce test cases for the different variants
• A grouped and local variation of the activity implemented in one of the engines.