Spring Batch Boot - Write data to multiple places based on the Status Code while reading data from CSV?

Question

I am developing Spring Boot Batch example. In this example, reading data from the CSV and based on status code values (like SUCCESS, REJECT, PENDING, COMPLETED) I want to write it into 4 different places (MySQL, XML, Postgres, Salesforce DB) while writing ?. I can see we can pass only 1 type of List of Customers to CompositeItemWriter method. How can I pass 4 different Customer List to CompositeItemWriter ?

Here I could think of using CompositeItemProcessor and CompositeItemWriter. In the CompositeItemProcessor I will take the decision of separating the records in all these processors and now question is how can I passed the SUCCESS to MYSQL, REJECT to XML, PENDING to Postgres and COMPLETED to Salesforce DB?

@Bean
public CompositeItemProcessor<Customer, Customer> compositeItemProcessor() throws Exception{
    List<ItemProcessor<Customer, Customer>> delegates = new ArrayList<>();
    delegates.add(new FilteringSuccessProcessor());
    delegates.add(new FilteringRejectProcessor());
    delegates.add(new FilteringPendingProcessor());
    delegates.add(new FilteringCompletedProcessor());

    CompositeItemProcessor<Customer, Customer> processor = new CompositeItemProcessor<>();
    processor.setDelegates(delegates);
    processor.afterPropertiesSet();

    return processor;
}

FilteringSuccessProcessor.java

Like this I've created processor all the Status Codes and

public class FilteringSuccessProcessor implements ItemProcessor<Customer, Customer> {

    @Override
    public Customer process(Customer item) throws Exception {
        if(item.getStatus == "SUCCESS"){
            return item;
        }
        else
            return null;
    }
}

Now my question is - How can we pass 4 different data results to CompositeItemWriter to write it into 4 different places?

I though of using CustomerClassifier as well, but how to pass four different list to same writer ?

Any guidance ? This is very interesting use case, but look like critical to implement.

I second the answer by @slimane, the `ClassifierCompositeItemWriter` is the way to go. A similar answer with an example here: https://stackoverflow.com/a/53388876/5019386 — Mahmoud Ben Hassine, Dec 19 '18 at 21:54

stacker · Accepted Answer · 2018-12-19T20:17:56.657

you can use classifierCompositeItemWriter and define your routing policy inside it:

it is an ItemWriter that will take care of writing to different places base on your status using a Classifier.

    @Bean
    public ClassifierCompositeItemWriter<Item> classifierCompositeItemWriter(ItemWriter<Item> POSTGRESWriter, ItemWriter<Item> SQLWiter) {
        ClassifierCompositeItemWriter<Item> classifierCompositeItemWriter = new ClassifierCompositeItemWriter<>();
        classifierCompositeItemWriter.setClassifier((Classifier<Item, ItemWriter<? super Item>>) item-> {
            if (item.getStatus("Success")) {
                return POSTGRESWriter;
            } else {
                return SQLWiter;
            }
        });
        return classifierCompositeItemWriter;
    }

you can configure your Step as Follow:

    @Bean
    public Step process() {
        return stepBuilderFactory.get("dataExtraprocessctionStep")
                .<Item, Item>chunk(1)
                .reader(itemReader())
                .writer(classifierCompositeItemWriter(POSTGRESWriter(), SQLWriter()))
                .stream(POSTGRESWriter())
                .stream(SQLWriter())
                .build();
    }

I am not clear on the return POSTGRESWriter. Can we return instance of ItemWriter from ClassifierCompositeItemWriter. Could you please elaborate your answer ? — PAA, Dec 19 '18 at 20:04
I second this answer, the `ClassifierCompositeItemWriter` is the way to go. A similar answer with an example here: https://stackoverflow.com/a/53388876/5019386 — Mahmoud Ben Hassine, Dec 19 '18 at 21:57

Spring Batch Boot - Write data to multiple places based on the Status Code while reading data from CSV?

1 Answers1