1. 申请Application
1) 连接ResourceManager
YarnConfiguration yarnConf = new YarnConfiguration(conf);
InetSocketAddress rmAddress = NetUtils.createSocketAddr(yarnConf.get(
YarnConfiguration.RM_ADDRESS,
YarnConfiguration.DEFAULT_RM_ADDRESS));
LOG.info("Connecting to ResourceManager at " rmAddress);
ClientRMProtocol applicationsManager = ((ClientRMProtocol) rpc.getProxy(
ClientRMProtocol.class, rmAddress, conf));
2) 向ResourceManager申请一个Application
GetNewApplicationRequest request = Records.newRecord(GetNewApplicationRequest.class);
GetNewApplicationResponse response = applicationsManager.getNewApplication(request);
LOG.info("Got new application id=" response.getApplicationId());
2. 初始化ApplicationMaster的上下文(
ApplicationId
ApplicationName
Queue:Application将被提交到的队列
Priority:Application的优先级
User:运行该Application的用户
AMContainerSpec:运行ApplicationMaster的Container的信息
)
1) 设置Application属性
ApplicationId appId = response.getApplicationId();
ApplicationSubmissionContext appContext = Records.newRecord(ApplicationSubmissionContext.class);
appContext.setApplicationId(appId);
appContext.setApplicationName(appName);
2) 设置运行ApplicationMaster的Container的上下文
ContainerLaunchContext amContainer = Records.newRecord(ContainerLaunchContext.class);
// 上传包含ApplicationMaster的jar包
FileSystem fs = FileSystem.get(conf);
Path src = new Path(appMasterJar);
String pathSuffix = appName "/" appId.getId() "/AppMaster.jar";
Path dst = new Path(fs.getHomeDirectory(), pathSuffix);
fs.copyFromLocalFile(false, true, src, dst);
FileStatus destStatus = fs.getFileStatus(dst);
// 上传ApplicationMaster的log4j配置文件
Path log4jSrc = new Path(log4jPropFile);
Path log4jDst = new Path(fs.getHomeDirectory(), "log4j.props");
fs.copyFromLocalFile(false, true, log4jSrc, log4jDst);
FileStatus log4jFileStatus = fs.getFileStatus(log4jDst);
// 设置LocalResources
LocalResource amJarRsrc = Records.newRecord(LocalResource.class);
amJarRsrc.setType(LocalResourceType.FILE);
amJarRsrc.setVisibility(LocalResourceVisibility.APPLICATION);
amJarRsrc.setResource(ConverterUtils.getYarnUrlFromPath(dst));
amJarRsrc.setTimestamp(destStatus.getModificationTime());
amJarRsrc.setSize(destStatus.getLen());
Map<String, LocalResource> localResources = new HashMap<String, LocalResource>();
localResources.put("AppMaster.jar", amJarRsrc);
LocalResource log4jRsrc = Records.newRecord(LocalResource.class);
log4jRsrc.setType(LocalResourceType.FILE);
log4jRsrc.setVisibility(LocalResourceVisibility.APPLICATION);
log4jRsrc.setResource(ConverterUtils.getYarnUrlFromURI(log4jDst.toUri()));
log4jRsrc.setTimestamp(log4jFileStatus.getModificationTime());
log4jRsrc.setSize(log4jFileStatus.getLen());
localResources.put("log4j.properties", log4jRsrc);
amContainer.setLocalResources(localResources);
// 设置运行ApplicationMaster的环境变量
StringBuilder classPathEnv = new StringBuilder("${CLASSPATH}:./*");
for (String c : conf.get(YarnConfiguration.YARN_APPLICATION_CLASSPATH).split(",")) {
classPathEnv.append(':');
classPathEnv.append(c.trim());
}
classPathEnv.append(":./log4j.properties");
Map<String, String> env = new HashMap<String, String>();
env.put("CLASSPATH", classPathEnv.toString());
amContainer.setEnvironment(env);
// 设置运行ApplicationMaster的命令
Vector<CharSequence> vargs = new Vector<CharSequence>(30);
vargs.add("${JAVA_HOME}" + "/bin/java");
vargs.add("-Xmx" + amMemory + "m");
vargs.add(appMasterMainClass);
vargs.add(some args);
vargs.add("1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/AppMaster.stdout");
vargs.add("2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/AppMaster.stderr");
StringBuilder command = new StringBuilder();
for (CharSequence str : vargs) {
command.append(str).append(" ");
}
List<String> commands = new ArrayList<String>();
commands.add(command.toString());
amContainer.setCommands(commands);
// 设置ApplicationMaster运行所需的内存
Resource capability = Records.newRecord(Resource.class);
capability.setMemory(amMemory);
amContainer.setResource(capability);
appContext.setAMContainerSpec(amContainer);
3) 设置优先级、队列、用户等
Priority pri = Records.newRecord(Priority.class);
pri.setPriority(amPriority);
appContext.setPriority(pri);
appContext.setQueue(amQueue);
appContext.setUser(amUser);
3. 创建运行application的request,并提交
SubmitApplicationRequest appRequest = Records.newRecord(SubmitApplicationRequest.class);
appRequest.setApplicationSubmissionContext(appContext);
applicationsManager.submitApplication(appRequest);
4. 向ResourceManager轮训Application的状态
1) 构建获取Application状态的request并提交
GetApplicationReportRequest reportRequest = Records.newRecord(GetApplicationReportRequest.class);
reportRequest.setApplicationId(appId);
GetApplicationReportResponse reportResponse = applicationsManager.getApplicationReport(reportRequest);
ApplicationReport report = reportResponse.getApplicationReport();
2) 获取Application的相关信息
LOG.info("Got application report from ASM for"
+ ", appId=" + appId.getId()
+ ", clientToken=" + report.getClientToken()
+ ", appDiagnostics=" + report.getDiagnostics()
+ ", appMasterHost=" + report.getHost()
+ ", appQueue=" + report.getQueue()
+ ", appMasterRpcPort=" + report.getRpcPort()
+ ", appStartTime=" + report.getStartTime()
+ ", yarnAppState=" + report.getYarnApplicationState().toString()
+ ", distributedFinalState=" + report.getFinalApplicationStatus().toString()
+ ", appTrackingUrl=" + report.getTrackingUrl()
+ ", appUser=" + report.getUser());
YarnApplicationState state = report.getYarnApplicationState();
FinalApplicationStatus dsStatus = report.getFinalApplicationStatus();
分享到:
相关推荐
oracle客户端详细的安装步骤!
自己的安装步骤 希望对大家有用。环境是:主机XP系统,虚拟机中安装Centos系统,Oracle 11g安装在XP中,Oracle client 11g安装在Linux中。通过client连接主机Oracle。
Freeradius Server 以及 Freeradius Client配置及测试详细步骤,在Centos 6.x操作系统上可以完全按照步骤进行配置和测试等相关操作,能够实现radius服务器的搭建,并且使用客户端进行验证测试。另外,附带了整理出来...
TCP/IP socket代码,server等待客户端链接接收参数,client启动发送参数等待server返回 启动server端等待client端发送一个域名过来,把ping结果传回客户端 启动client端需要带一个域名参数
32位Ubuntu12.04.2LTS安装iNodeClient详细步骤及问题解决 各种软链接问题、各种“not found”问题
RTSP 流媒体,rtsp client 测试例程
S7ClientDemo一些PLC相关的功能讲解与应用
MacOS下安装Genero Desktop Client及参数配置方法
单机版可以在电脑上同时安装oracle11g和 10201_client_win32.zip 服务器版可以在本机安装10g客户端
Kepware使用教程—OPC Quick Client具体使用 ;希望对初入Kepware的朋友有很好的帮助
本《适用于 Linux 的 VMware Horizon Client 安装和设置指南》...和支持的操作系统由供应商、型号以及企业选择使用的配置决定。有关这些客户端设备的供应商和型号的 信息,请参阅 VMware 网站上的 VMware 兼容性指南。
clientDEMO,监控必备软件,监控调试助手,网络必备 clientDEMO,监控必备软件,监控调试助手,网络必备
Microsoft SQL Server Native Client (SQL Native Client) 是一个同时包含 SQL OLE DB 访问接口和 SQL ODBC 驱动程序的动态链接库 (DLL)。它对使用本机代码 API(ODBC、OLE DB 和 ADO)连接到 Microsoft SQL Server ...
在上文Docker源码分析之——Docker Daemon的启动中,介绍了DockerDaemon进程的启动。DockerDaemon可以认为是一个Docker作为Server的运行载体,而真正发送关于docker ...DockerClient启动的流程与DockerDae
Arctium WoW Client Launcher 27377版本。 HOWTO: 安装方法: 1.Use 7zip/WinRAR to extract the archive ...2.双击Arctium WoW Client Launcher.exe,等待运行完毕,程序会自动启动wow-64.exe 3.Profit! 3.开玩!
工具:win32_11gR2_client 方法/步骤: 安装包里点击setup.exe可执行程序,选择“运行时”点下一步 选择简体中文点下一步 选择要安装的路径点下一步 执行“先决条件检查”的过程中,有可能会提示PATH路径有问题,...
M1.Client.rarM1.Client.rarM1.Client.rarM1.Client.rarM1.Client.rarM1.Client.rarM1.Client.rarM1.Client.rarM1.Client.rarM1.Client.rarM1.Client.rarM1.Client.rarM1.Client.rarM1.Client.rarM1.Client.rar
kettle生成经纬度, REST Client、Http Client组件例子
Oracle Instant Client windows大全 包含32位64位各版本 instantclient-basic-win32-10.1.0.5-20060419 instantclient-basic-win32-10.2.0.5 instantclient-basic-win32-11.1.0.7.0 instantclient-basic-nt-11.2....
华为工具SVNClient华为SVNclient特意分享出来