Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The HDFS filesystem should support re-trying blocks with datanode failure #4

Open
tarnfeld opened this issue Nov 12, 2014 · 0 comments
Labels

Comments

@tarnfeld
Copy link
Contributor

If between the point the client requests a block location from the namenode, the datanode it was given drops out of the cluster, an exception is bubbled to the top... this exception is a result of us not being able to connect to the datanode to get the block.

In this specific case, we should detect a datanode failure and ask the namenode again for a location of the block. I'm not sure how many times we should retry this process, I think the hadoop-client does it indefinitely.

Let's consult the java implementation and follow that.

Note: this filesystem implementation is still sitting in review via #2

@tarnfeld tarnfeld added the bug label Nov 12, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant