Hi,
I followed KB http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=209105... in order to add HDP 2.1.1
I had an error at this step (deploy basic cluster ) :
Logs:
Chef::Exceptions::Package: package[hadoop] (hadoop_cluster::default line 363) had an error: Chef::Exceptions::Package: No version specified, and no candidate version available for hadoop
My repo seems ok (hadoop repository):
Any help would be very appreciate
Could you attach the full ironfan.log and /opt/serengeti/www/distros/manifest on the bde server ?
The issue you met is not related to 2.1.1. But I strongly suggest use the latest HDP 2.1.10. The only difference between 2.1.1 and 2.1.10 is the step 4 in the KB VMware KB: Adding the Hortonworks Data Platform (HDP) Distribution for Hadoop for use with VMware vS...,
4. Enter these information in the new repo file and save the file:
[hortonworks-hdp2]
name=Hortonworks Data Platform Version - HDP-2.1.10.0
baseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.10.0
gpgcheck=1
enabled=1
priority=1
did you run this command when creating the yum repo? You can send the full /opt/Serengeti/logs/ironfan.log for us to debug.
BTW, hdp 2.1.1 has a bug and cannot work with bde. You have to use 2.1.2+. Hdp 2.2 is not supported by default app manager, but supported by ambari app manager in BDE 2.2 released recently.
EDIT: Will try with 2.1.15
Hi Jessie,
Yes I created and configured the repository.
About ironfan.log :
10.197.177.160 [2015-07-07T11:36:48+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)^M
Bootstrapping node HDP-Worker-0 (10.197.177.160) completed with exit status 1
[2015-07-07T13:36:48+02:00] INFO: Monitoring bootstrap progress of cluster HDP with data: [1, <Ironfan::Common::Server {"name"=>"HDP-Worker-0", "chef_attributes"=>{}, "instances"=>1, "environment"=>:_default, "hadoop_distro"=>"BDE_HortonWorks", "run_list"=>["role[hadoop_datanode]", "role[hadoop_nodemanager]", "role[HDP-cluster]", "role[HDP-Worker-facet]"] chef: HDP-Worker-0 fog: }>]
10.197.177.161 ^M
10.197.177.161 ^[[31m* No version specified, and no candidate version available for hadoop^[[0m^[[0m^M
10.197.177.161 ================================================================================^[[0m^M
10.197.177.161 ^[[31mError executing action `install` on resource 'package[hadoop]'^[[0m^M
10.197.177.161 ================================================================================^[[0m^M
10.197.177.161 ^M
10.197.177.161 ^[[0m Chef::Exceptions::Package^[[0m^M
10.197.177.161 -------------------------^[[0m^M
10.197.177.161 No version specified, and no candidate version available for hadoop^[[0m^M
10.197.177.161 ^M
10.197.177.161 ^[[0m Resource Declaration:^[[0m^M
10.197.177.161 ---------------------^[[0m^M
10.197.177.161 # In /var/chef/cache/cookbooks/hadoop_cluster/libraries/hadoop_cluster.rb^M
10.197.177.161 ^[[0m ^M
10.197.177.161 ^[[0m 363: package name do^M
10.197.177.161 ^[[0m 364: # Add retry for basic hadoop rpms which have big file size (about 20M).^M
10.197.177.161 ^[[0m 365: # The yum install command might timeout when the yum server is overloaded.^M
10.197.177.161 ^[[0m 366: retries 6^M
10.197.177.161 ^[[0m 367: retry_delay 5^M
10.197.177.161 ^[[0m 368: ^M
10.197.177.161 ^[[0m 369: if node[:hadoop][:package_version] != 'current'^M
10.197.177.161 ^[[0m 370: version node[:hadoop][:package_version]^M
10.197.177.161 ^[[0m 371: end^M
10.197.177.161 ^[[0m 372: end^M
10.197.177.161 ^[[0m ^M
10.197.177.161 ^[[0m Compiled Resource:^[[0m^M
10.197.177.161 ------------------^[[0m^M
10.197.177.161 # Declared in /var/chef/cache/cookbooks/hadoop_cluster/libraries/hadoop_cluster.rb:363:in `block in hadoop_package'^M
10.197.177.161 ^[[0m ^M
10.197.177.161 ^[[0m package("hadoop") do^M
10.197.177.161 ^[[0m action :install^M
10.197.177.161 ^[[0m retries 0^M
10.197.177.161 ^[[0m retry_delay 5^M
10.197.177.161 ^[[0m guard_interpreter :default^M
10.197.177.161 ^[[0m package_name "hadoop"^M
10.197.177.161 ^[[0m timeout 900^M
10.197.177.161 ^[[0m cookbook_name "hadoop_cluster"^M
10.197.177.161 ^[[0m recipe_name "default"^M
10.197.177.161 ^[[0m end^M
10.197.177.161 ^[[0m ^M
10.197.177.161 ^[[0m[2015-07-07T11:36:48.917+0000] INFO: Running queued delayed notifications before re-raising exception^M
10.197.177.161 ^[[0m^M
10.197.177.161 Running handlers:^[[0m^M
10.197.177.161 [2015-07-07T11:36:48.918+0000] ERROR: Running exception handlers^M
10.197.177.161 [2015-07-07T11:36:48.918+0000] FATAL: Cannot install package hadoop on node HDP-Client-0. SSH to this node and run the command 'sudo yum install hadoop -y' to view error messages.^M
I wasn't aware of this bug between hdp 2.1.1 and BDE 2.1, do you have more information ?
Did you succeed to deploy HDP 2.1.2 with default app manager ? If yes which repository did you use ?
Could you attach the full ironfan.log and /opt/serengeti/www/distros/manifest on the bde server ?
The issue you met is not related to 2.1.1. But I strongly suggest use the latest HDP 2.1.10. The only difference between 2.1.1 and 2.1.10 is the step 4 in the KB VMware KB: Adding the Hortonworks Data Platform (HDP) Distribution for Hadoop for use with VMware vS...,
4. Enter these information in the new repo file and save the file:
[hortonworks-hdp2]
name=Hortonworks Data Platform Version - HDP-2.1.10.0
baseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.10.0
gpgcheck=1
enabled=1
priority=1
I tried with release 2.1.15 and now it works
baseurl:http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.15.0
Maybe my repo was corrupt :smileyconfused: because my previous manifest for 2.1.1 seems ok :
"name": "BDE_HortonWorks",
"vendor": "HDP",
"version": "2.1.1",
"packages": [
{
"package_repos": [
"http://10.197.177.180/hdp/2/hortonworks-hdp2.repo"
],
"roles": [
"hadoop_namenode",
"hadoop_datanode",
"hadoop_resourcemanager",
"hadoop_nodemanager",
"hadoop_journalnode",
"hadoop_client",
"hive",
"hive_server",
"pig",
"hbase_master",
"hbase_regionserver",
"hbase_client",
"zookeeper"
]
}
]
},
Glad to know it. "version": "2.1.1" can be updated to 2.1.15 although it doesn't impact any function. I will update the KB to use 2.1.15 accordingly.
Are you using BDE 2.1 or 2.2 which can also deploy HDP cluster via Ambari ? Or you just want to use the Chef based Default App Manager in BDE?