the question I got might be little general and so not the best for StackOverflow- sorry for that. However, I'm googleing for answer and not finding any.
In our DWH project (AWS S3 + Redshift + informatica) we have hundreds of ETL jobs. Each ETL job was designed by analyst and developed by ETL developer. Analysts are creating excel documents describing high level business functionality.We have feeling that we reach the point that Excel based documentation start causing issues. We need tool that allow us to:
- Define ETLs on field level trnsformation but not require strict development (we need documentation tool after all rather than parallel ETL development environment)
- Present metadata in graphical way
- allow versioning of metadata and high-lining changes (e.g. Analyst performed modification of one transformation in ETL docummentation and pass to ETL developer)
- cross technological lineage- ability to click on field in Data Mart (or even measure in BI tool) and present all it dependencies and transformations till source system
- Impact analysis- ability to click on source field and see each fields in data marts that will be affected
It looks to me that ERWIN Data Intelligence will do the job. I'm wandering if there are any similar products on the market that we should consider?
