This project includes a group of scripts which can deploy a hadoop cluster on docker containers automatically, including how to use docker to create a container, how to use weave to connect two physical machines, how to specify the number of cpu cores, cpu frequency, memory, and disk io speed for a container, etc. Also, it can run different mapreduce applications on clusters and collect useful information from hadoop logs automatically. Finally, these useful data will be fed to a tensorflow DNN model to learn and give a prediction for a task's execution time
-
Notifications
You must be signed in to change notification settings - Fork 0
This project includes a group of scripts which can deploy a hadoop cluster on docker containers automatically, including how to use docker to create a container, how to use weave to connect two physical machines, how to specify the number of cpu cores, cpu frequency, memory, and disk io speed for a container, etc. Also, it can run different mapr…
lyuxiaosu/Hadoop-cluster-automation-deployment.
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
This project includes a group of scripts which can deploy a hadoop cluster on docker containers automatically, including how to use docker to create a container, how to use weave to connect two physical machines, how to specify the number of cpu cores, cpu frequency, memory, and disk io speed for a container, etc. Also, it can run different mapr…
Resources
Stars
Watchers
Forks
Packages 0
No packages published