IssueTracker

This will create a public issue which anybody can view and comment on.

Please provide as much information as possible. At least, this should include a description of your issue and steps to reproduce the problem. If possible please provide a summary of what steps or workarounds you have already tried, and any docs or articles you found (un)helpful.

Problem you have encountered:

• Error: Data Transfer between remote projects gets terminated with an Early EOF

• When transferring data from GCP remote VM to Dataproc Solr. This exception terminates the data load into the cluster and data is partially loaded, only 30-40% of the data records are loaded.

• While trying to load the data from inside the SOLR Project master nodes, there are no issues.

• There appears to be a tooling limitation with regards to uploading files to the cluster externally.

• There is a size limitation when upload larger files (greater than 50MB) through the UI and API

What you expected to happen:

• Explore improving the tooling functionality for external Dataproc cluster uploads.
• Explore the possibility of adding functionality for uploading larger files (greater than 50MB) through the use of UI or API (using curl command)

Steps to reproduce:

• Tested and the max cap is at ~50 MB. If the file size is greater than 50 MB then socket connection is terminated causing Early EOF error in logs which we see in the user tarball as well.

Other information (workarounds you have tried, documentation consulted, etc):

• Possible workaround, try uploading files in batches of 50MB or less in size (if uploading from outside of the cluster)