We will try to execute Dexecutor in a distributed mode using Apache Ignite. For the demo we would be setting up multiple Ignite nodes on single machine.
Refer Introducing Dexecutor, to get an introduction on Dexecutor and to understand the problem we would solve in a distribute fashion. In short:
We would be distributing the execution of dexecutor tasks on Apache Ignite nodes in a single machine.
To do that one of the nodes would act as master and submit the tasks to Ignite to be executed by other Ignite compute nodes using Dexecutor.
Here are the steps to do that :
Step 1: Add dexecutor-ignite dependency
<dependency> <groupId>com.github.dexecutor<groupId> <artifactId>dexecutor-ignite<artifactId> <version>LATEST_RELEASE<version> <dependency>
Step 2: Start Ignite
IgniteConfiguration cfg = new IgniteConfiguration(); cfg.setGridName(nodeName); Ignite ignite = Ignition.start(cfg);
Step 3 : Create Dexecutor using Ignite
if (isMaster) { DefaultDependentTasksExecutor<Integer, Integer> dexecutor = newTaskExecutor(ignite.compute()); buildGraph(dexecutor); dexecutor.execute(ExecutionConfig.TERMINATING); }
private DefaultDependentTasksExecutor<Integer, Integer> newTaskExecutor(final IgniteCompute igniteCompute) { DependentTasksExecutorConfig<Integer, Integer> config = new DependentTasksExecutorConfig<Integer, Integer>( new IgniteExecutionEngine<Integer, Integer>(igniteCompute), new SleepyTaskProvider()); return new DefaultDependentTasksExecutor<Integer, Integer>(config); }
Step 4: Execution
Open three terminals and execute the following :
Terminal #1
mvn test-compile exec:java -Djava.net.preferIPv4Stack=true -Dexec.mainClass="com.github.dexecutor.ignite.Node" -Dexec.classpathScope="test" -Dexec.args="s node-A"
Terminal #2
mvn test-compile exec:java -Djava.net.preferIPv4Stack=true -Dexec.mainClass="com.github.dexecutor.ignite.Node" -Dexec.classpathScope="test" -Dexec.args="s node-B"
Terminal #3
mvn test-compile exec:java -Dexec.classpathScope="test" -Djava.net.preferIPv4Stack=true -Dexec.mainClass="com.github.dexecutor.ignite.Node" -Dexec.args="m node-C"
Here is the Node Implementation.
One thought on “Multi Node Distributed Execution Using Ignite And Dexecutor”