Improve Error Handling in Local Directory Feeds This change improves handling of two error types for filesystem based feeds. The first one is the handling of IO Errors which causes the input stream to be closed, and the second one is reacting to missed filesystem events. In both cases, we scan the directory and compare it with the history we have in order to resume from where we last left off.
In addition, this change includes some refactoring in external data. Particularly, we get rid of the stream provider layer and instead, stream factories create input streams directly. This is consistent with record reader factories which create readers directly without reader providers.
Support Change Feeds and Ingestion of Records with MetaData This change allows feeds to perform upserts and deletes in order to perform replication of an external data source. The change does so by performing the following: 1. The adapter produces [PK][Record]. (Record == null --> delete) 2. The insert is replaced by an upsert operator.