Skip to content

Commit

Permalink
feat: 添加 EMRInstanceCredentialsProvider 到鉴权身份获取列表中
Browse files Browse the repository at this point in the history
  • Loading branch information
@yuyang733
yuyang733 committed Jan 19, 2022
1 parent 5d3f406 commit 63b20f1
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 3 deletions.
8 changes: 5 additions & 3 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,10 @@ done
1. org.apache.hadoop.fs.auth.SessionCredentialProvider
2. org.apache.hadoop.fs.auth.SimpleCredentialProvider
3. org.apache.hadoop.fs.auth.EnvironmentVariableCredentialProvider
4. org.apache.hadoop.fs.auth.CVMInstanceCredentialsProvider
5. org.apache.hadoop.fs.auth.CPMInstanceCredentialsProvider
4. org.apache.hadoop.fs.auth.SessionTokenCredentialProvider
5. org.apache.hadoop.fs.auth.CVMInstanceCredentialsProvider
6. org.apache.hadoop.fs.auth.CPMInstanceCredentialsProvider
7. org.apache.hadoop.fs.auth.EMRInstanceCredentialsProvider
</description>
</property>

Expand Down Expand Up @@ -199,7 +201,7 @@ done
| 属性键 | 说明 | 默认值 | 必填项 |
|:-----------------------------------:|:--------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:---:|
|fs.defaultFS | 配置hadoop默认使用的底层文件系统，如果想使用cos作为hadoop默认文件系统，则此项应设置为cosn://bucket-appid，此时可以通过文件路径访问cos对象，如/hadoop/inputdata/test.dat。若不想把cos作为hadoop默认文件系统，则不需要修改此项，当需要访问cos上的对象时，则指定完整的uri即可，如cosn://testbucket-1252681927/hadoop/inputdata/test.dat来访问。|
|fs.cosn.credentials.provider |配置secret id和secret key的获取方式。当前支持三种获取方式：1.org.apache.hadoop.fs.auth.SessionCredentialProvider：从请求URI中获取secret id和secret key，其格式为：cosn://{secretId}:{secretKey}@examplebucket-1250000000000/; 2.org.apache.hadoop.fs.auth.SimpleCredentialProvider：从core-site.xml配置文件中读取fs.cosn.userinfo.secretId和fs.cosn.userinfo.secretKey来获取secret id和secret key; 3.org.apache.hadoop.fs.auth.EnvironmentVariableCredentialProvider：从系统环境变量COS_SECRET_ID和COS_SECRET_KEY中获取；4.org.apache.hadoop.fs.auth.SessionTokenCredentialProvider: 设置token；5.org.apache.hadoop.fs.auth.CVMInstanceCredentialsProvider：利用腾讯云云服务器（CVM）绑定的角色，获取访问COS的临时密钥; 6. org.apache.hadoop.fs.auth.CPMInstanceCredentialsProvider：利用腾讯云黑石物理机（CPM）绑定的角色，获取访问COS的临时密钥。7. org.apache.hadoop.fs.auth.RangerCredentialsProvider 使用ranger进行获取秘钥 | 如果不指定改配置项，默认会按照以下顺序读取：1.org.apache.hadoop.fs.auth.SessionCredentialProvider; 2.org.apache.hadoop.fs.auth.SimpleCredentialProvider;3.org.apache.hadoop.fs.auth.EnvironmentVariableCredentialProvider;4.org.apache.hadoop.fs.auth.SessionTokenCredentialProvider；5.org.apache.hadoop.fs.auth.CVMInstanceCredentialsProvider;6.org.apache.hadoop.fs.auth.CPMInstanceCredentialsProvider |否|
|fs.cosn.credentials.provider |配置secret id和secret key的获取方式。当前已经支持八种获取方式：1.org.apache.hadoop.fs.auth.SessionCredentialProvider：从请求URI中获取secret id和secret key，其格式为：cosn://{secretId}:{secretKey}@examplebucket-1250000000000/； 2.org.apache.hadoop.fs.auth.SimpleCredentialProvider：从core-site.xml配置文件中读取fs.cosn.userinfo.secretId和fs.cosn.userinfo.secretKey来获取secret id和secret key； 3.org.apache.hadoop.fs.auth.EnvironmentVariableCredentialProvider：从系统环境变量COS_SECRET_ID和COS_SECRET_KEY中获取；4.org.apache.hadoop.fs.auth.SessionTokenCredentialProvider: 设置token；5.org.apache.hadoop.fs.auth.CVMInstanceCredentialsProvider：利用腾讯云云服务器（CVM）绑定的角色，获取访问 COS 的临时密钥； 6.org.apache.hadoop.fs.auth.CPMInstanceCredentialsProvider：利用腾讯云黑石物理机（CPM）绑定的角色，获取访问 COS 的临时密钥；7.org.apache.hadoop.fs.auth.EMRInstanceCredentialsProvider：利用腾讯云 EMR 实例绑定的角色，获取访问 COS 的临时密钥；8.org.apache.hadoop.fs.auth.RangerCredentialsProvider 使用 ranger 进行获取秘钥。 | 如果不指定改配置项，默认会按照以下顺序读取： 1.org.apache.hadoop.fs.auth.SessionCredentialProvider; 2.org.apache.hadoop.fs.auth.SimpleCredentialProvider; 3.org.apache.hadoop.fs.auth.EnvironmentVariableCredentialProvider; 4.org.apache.hadoop.fs.auth.SessionTokenCredentialProvider; 5.org.apache.hadoop.fs.auth.CVMInstanceCredentialsProvider; 6.org.apache.hadoop.fs.auth.CPMInstanceCredentialsProvider; 7.org.apache.hadoop.fs.auth.EMRInstanceCredentialsProvider; |否|
|fs.cosn.useHttps|配置是否使用https协议。| false |否|
|fs.cosn.bucket.endpoint_suffix|指定要连接的COS endpoint，该项为非必填项目。对于公有云COS用户而言，只需要正确填写上述的region配置即可。兼容原配置项：fs.cosn.userinfo.endpoint_suffix。| 无 |否|
|fs.cosn.userinfo.secretId/secretKey| 填写您账户的API 密钥信息。可通过 [云 API 密钥 控制台](https://console.cloud.tencent.com/capi) 查看。| 无 | 是|
Expand Down
1 change: 1 addition & 0 deletions src/main/java/org/apache/hadoop/fs/CosNUtils.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ public static COSCredentialProviderList createCosCredentialsProviderSet(
credentialProviderList.add(new SessionTokenCredentialProvider(uri,conf));
credentialProviderList.add(new CVMInstanceCredentialsProvider(uri, conf));
credentialProviderList.add(new CPMInstanceCredentialsProvider(uri, conf));
credentialProviderList.add(new EMRInstanceCredentialsProvider(uri, conf));
} else {
for (Class<?> credClass : cosClasses) {
credentialProviderList.add(createCOSCredentialProvider(uri,
Expand Down

0 comments on commit 63b20f1

Please sign in to comment.