Step 5: Install and configure Nova

Submitted by martin on Fri, 2012-03-23 22:35

OpenStack Compute, codenamed Nova, is by far the most important and the most substantial openstack component. Whatever you do when it comes to managing VMs will be done by Nova in the background. The good news is: Nova is basically controlled by one configuration file, /etc/nova/nova.conf. Get started by installing all nova-related components:

apt-get install nova-api nova-cert nova-common nova-compute nova-compute-kvm nova-doc nova-network nova-objectstore nova-scheduler nova-volume nova-consoleauth novnc python-nova python-novaclient

Then, open /etc/nova/nova.conf and replace everything in there with these lines:

--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/var/lock/nova
--allow_admin_api=true
--use_deprecated_auth=false
--auth_strategy=keystone
--scheduler_driver=nova.scheduler.simple.SimpleScheduler
--s3_host=10.42.0.6
--ec2_host=10.42.0.6
--rabbit_host=10.42.0.6
--cc_host=10.42.0.6
--nova_url=http://10.42.0.6:8774/v1.1/
--routing_source_ip=10.42.0.6
--glance_api_servers=10.42.0.6:9292
--image_service=nova.image.glance.GlanceImageService
--iscsi_ip_prefix=192.168.22
--sql_connection=mysql://novadbadmin:dieD9Mie@10.42.0.6/nova
--ec2_url=http://10.42.0.6:8773/services/Cloud
--keystone_ec2_url=http://10.42.0.6:5000/v2.0/ec2tokens
--api_paste_config=/etc/nova/api-paste.ini
--libvirt_type=kvm
--libvirt_use_virtio_for_bridges=true
--start_guests_on_host_boot=true
--resume_guests_state_on_host_boot=true
--vnc_enabled=true
--vncproxy_url=http://10.42.0.6:6080
--vnc_console_proxy_url=http://10.42.0.6:6080
# network specific settings
--network_manager=nova.network.manager.FlatDHCPManager
--public_interface=eth0
--flat_interface=eth1
--flat_network_bridge=br100
--fixed_range=192.168.22.32/27
--floating_range=10.42.0.32/27 
--network_size=32
--flat_network_dhcp_start=192.168.22.33
--flat_injected=False
--force_dhcp_release
--iscsi_helper=tgtadm
--connection_type=libvirt
--root_helper=sudo nova-rootwrap
--verbose
--libvirt_use_virtio_for_bridges
--ec2_private_dns_show
--novnc_enabled=true
--novncproxy_base_url=http://10.42.0.6:6080/vnc_auto.html
--vncserver_proxyclient_address=10.42.0.6
--vncserver_listen=10.42.0.6

As you can see, many of the entries in this file are self-explanatory; the trickiest bit to get done right is the network configuration part, which you can see at the end of the file. We're using Nova's FlatDHCP network mode; 192.168.22.32/27 is the fixed range from which our future VMs will get their IP adresses, starting with 192.168.22.33. Our flat interface is eth1 (nova-network will bridge this into a bridge named br100), our public interface is eth0. An additional floating range is defined at 10.42.0.32/27 (for those VMs that we want to have a 'public IP').

Attention: Every occurance of 10.42.0.6 in this file refers to the IP of the machine I used for writing this guide. You need to replace it with the actual machine IP of the box you are running this on. For example, if your machine has the local IP address 192.168.0.1, then use this IP instead of 10.42.0.6.

After saving nova.conf, open /etc/nova/api-paste.ini in an editor and scroll down to the end of the file. Adapt it according to the changes you conducted in Glance's paste-files in step 3. Use service as tenant name and nova as username.

Then, restart all nova services to make the configuration file changes take effect:

for a in libvirt-bin nova-network nova-compute nova-cert nova-api nova-objectstore nova-scheduler nova-volume novnc nova-consoleauth; do service "$a" stop; done

for a in libvirt-bin nova-network nova-compute nova-cert nova-api nova-objectstore nova-scheduler nova-volume novnc nova-consoleauth; do service "$a" start; done

The next step will create all databases Nova needs in MySQL. While we are at it, we can also create the network we want to use for our VMs in the Nova databases. Do this:

nova-manage db sync

nova-manage network create private --fixed_range_v4=192.168.22.32/27 --num_networks=1 --bridge=br100 --bridge_interface=eth1 --network_size=32

Also, make sure that all files in /etc/nova belong to the nova user and the nova group:

chown -R nova:nova /etc/nova

Then, restart all nova-related services again:

for a in libvirt-bin nova-network nova-compute nova-cert nova-api nova-objectstore nova-scheduler nova-volume novnc nova-consoleauth; do service "$a" stop; done

for a in libvirt-bin nova-network nova-compute nova-cert nova-api nova-objectstore nova-scheduler nova-volume novnc nova-consoleauth; do service "$a" start; done

You should now see all these nova-* processes when doing ps auxw. And you should be able to use the numerous nova commands. For example,

nova list

should give you a list of all currently running VMs (none, the list should be empty). And

nova image-list

should show a list of the image you uploaded to Glance in the step before. If that's the case, Nova is working as expected and you can carry on with starting your first VM.

Comments

VM on second compute node failes to get metadata

Submitted by dmazan on Mon, 2013-01-28 01:36.

Followed this article to successfully setup a single node Openstack, launch and connect to multiple VM's. Installed Nova on a second machine, copied the nova.conf file over, changed the vncserver_proxyclient_address and vncserver_listen parameters to point to the second node ip address. Also added my_ip pointing to the second node and network_host pointing to the original for good measure.

The second node shows up in nova-manage service list and, once a VM or two are running on the first node, a newly booted VM ends up on the second node. That VM boots and is reachable with ssh, but it can't access metadata at http://169.254.169.254. Therefore, it doesn't get the public key and rejects the ssh connection.

There is an iptables entry in the nova-network-PREROUTING chain mapping 169.254.169.254 to port 8775 and it gets used a bunch of times as each VM boots. But the console logs show the requests timing out.

I've also tried adding metadata_host=the first node which changes the iptables entry to point to the first node. Same result.

Basically everything works except VM metadata on a second compute node. What's the trick?

error when executing "nova image-list"

Submitted by rampi on Sat, 2012-11-03 06:20.

Hi,

At the step when I try "nova image-list", I get the following error :

rampi@ubuntu:~$ nova image-list

ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500)

"nova list" and "glance index" work fine

rampi@ubuntu:~$ nova list

+----+------+--------+----------+

+----+------+--------+----------+

rampi@ubuntu:~$ sudo glance index

ID Name Disk Format Container Format Size

------------------------------------ ------------------------------ -------------------- -------------------- --------------

d2e4280e-f33e-4213-868d-85254fb2edaa Ubuntu 12.04 cloudimg amd64 qcow2 ovf 233242624

rampi@ubuntu:~$

Any suggestions on how to overcome the error. Thanks for the guide it has been really helpful so far.

Also, wanted to add that in the nova-api.log, I am seeing Unrecognized Content-Type error

012-11-03 08:42:08 INFO nova.api.openstack.wsgi [req-f5df2e4d-33ad-48d7-8bec-cf317b141e99 48d3a65125b942668a830ee86dfa90cb 6a532af3f5fe4d63ac43f67c646398ae] GET http://192.168.1.2:8774/v2/6a532af3f5fe4d63ac43f67c646398ae/images/detail

2012-11-03 08:42:08 DEBUG nova.api.openstack.wsgi [req-f5df2e4d-33ad-48d7-8bec-cf317b141e99 48d3a65125b942668a830ee86dfa90cb 6a532af3f5fe4d63ac43f67c646398ae] Unrecognized Content-Type provided in request from (pid=23270) get_body /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:697

2012-11-03 08:42:08 ERROR nova.api.openstack [req-f5df2e4d-33ad-48d7-8bec-cf317b141e99 48d3a65125b942668a830ee86dfa90cb 6a532af3f5fe4d63ac43f67c646398ae] Caught error: The request returned 500 Internal Server Error

Error creating network

Submitted by Valbrand on Thu, 2012-10-18 21:00.

Hello. I'm getting the following error message when I try to put the command to create the network:

#nova-manage network create [...]

Command failed, please check log for more info

2012-10-18 15:53:46 CRITICAL nova [req-40a80068-123a-4b17-9755-99bbe4129922 None None] 'vlan_start'

I couldnt find any log entry in /var/log/nova/ matching the time I entered the command, so I'm pretty much out of information about the error. Thanks in advance

nova error

Submitted by nbenitez on Fri, 2012-09-21 16:20.

I have this error

nova list

Could not find any suitable endpoint. Correct region?

ERROR:

nova.conf

Submitted by rfz on Fri, 2012-05-04 19:01.

Hi Martin,

Great documentation on installing Openstack, a much welcome quick step guide.

I've completed an install on a multi-node deployment, and ran into some problems using the detailed nova.conf options:

--start_guests_on_host_boot=true: I've never been able to get this working correctly on diablo or essex, the problem occurs when you restart services on the compute nodes. They try to reboot server instances, and leave stale instances states on Openstack, and arguably force the service nodes into errors. I've disabled this option, and have had no issues with deleting or restarting services on the compute nodes.

In the end, --resume_guests_state_on_host_boot=true starts all instances (that were running) after a node shutdown/start.

Also for those of you running multi-nodes, and running stress testing/ heavy deploy and delete scenarios, I've also found that changing the RPC backend to carrot is much more stable This can be changed with the following in your nova.conf (on all nodes)

--rpc_backend=nova.rpc.impl_carrot

Moving the RPC backend, I was able to deploy 100 instances, with less than 1% error rate, compared to more than 50% error rate using the standard option.

Cheers,

rob

RabbitMQ security

Submitted by rfz on Fri, 2012-05-04 18:39.

By default, nova uses the default user/password from the rabbitmq server install (guest/guest). This should be changed, to some other password and/or user. It can be done with the following commands on your cloud controller:

rabbitmqctl change_password guest MYNEWPASSWORD

You need to update this change on all your nova.conf files, with the following line:

--rabbit_password=MYNEWPASSWORD

restart all your nova* services.

Hi rfz, thank you for all

Submitted by martin on Thu, 2012-05-10 14:09.

Hi rfz,

thank you for all your suggestions! :)

nova-volume

Submitted by Koop on Thu, 2012-04-19 00:10.

Extremely helpful tutorial! Thank you for taking the time to put this out there.

Nova

Submitted by banhbaokhin on Wed, 2012-04-18 03:53.

Hi Martin,

after using nova-manage db sync

i see this error

CRITICAL nova [-] (OperationalError) unable to open database file none none

That's strange, are you sure

Submitted by martin on Thu, 2012-05-10 14:31.

That's strange, are you sure you set up the MySQL path correctly in nova.conf?

RabbitMQ missing

Submitted by gmi on Tue, 2012-04-10 17:38.

There is no mention of installing rabbitmq-server, but it's definetly needed by Nova and it shows up in nova.conf

The "apt-get install" line should have the "rabbitmq-server" added as well.

RE: RabbitMQ missing

Submitted by martin on Mon, 2012-04-16 08:57.

Hi there,

Thanks for your feedback! rabbitmq-server gets installed in Step 1, "Prepare your system", already, so when people get to this part of the howto, RabbitMQ will already be there.

Best regards

Martin

2nd node

Submitted by Pandem on Mon, 2012-03-26 17:52.

Which changes needed in the nova.conf to setup 2nd and other next node?

Re: 2nd node

Submitted by martin on Mon, 2012-03-26 18:29.

Hi there,

I think you'll want to take a look at this document: http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-multiple-compute-nodes.html

Also, you'll have to specify the "--network_host=" parameter as explained in http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-flat-dhcp-networking.html (make sure it points to the host running nova-network).

Best regards

Martin

Extending your example to multi-node - issue with DHCP?

Submitted by sachinkagarwal on Fri, 2012-05-11 13:50.

Update:

Issue solved. My bad. Had not initialized eth1 in the new machine (Compute node) via the /etc/interfaces. So offcourse there was no network!

Dear Martin,

I tried extending your single node installation to a 2 node installation (Adding second compute node). I hope to use the same flatdhcp networking option. Both machines have 2 ethernet cards. So

1. installed Ubuntu 12.04 on the new machine

2. installed nova-compute on this machine

3. copied over the nova.conf from the first machien (public ip on eth1 is 192.168.1.10) to this new machine (public ip on eth0 is 192.168.1.12)

4. added

--network_host=192.168.1.10

Here is the nova.conf on the new machine

root@server2:/etc/nova# cat nova.conf

--dhcpbridge_flagfile=/etc/nova/nova.conf

--dhcpbridge=/usr/bin/nova-dhcpbridge

--logdir=/var/log/nova

--state_path=/var/lib/nova

--lock_path=/var/lock/nova

--allow_admin_api=true

--use_deprecated_auth=false

--auth_strategy=keystone

--scheduler_driver=nova.scheduler.simple.SimpleScheduler

--s3_host=192.168.1.10

--ec2_host=192.168.1.10

--rabbit_host=192.168.1.10

--cc_host=192.168.1.10

--nova_url=http://192.168.1.10:8774/v1.1/

--routing_source_ip=192.168.1.10

--glance_api_servers=192.168.1.10:9292

--image_service=nova.image.glance.GlanceImageService

--iscsi_ip_prefix=192.168.22

--sql_connection=mysql://novadbadmin:dieD9Mie@192.168.1.10/nova

--ec2_url=http://192.168.1.10:8773/services/Cloud

--keystone_ec2_url=http://192.168.1.10:5000/v2.0/ec2tokens

--api_paste_config=/etc/nova/api-paste.ini

--libvirt_type=kvm

--libvirt_use_virtio_for_bridges=true

--start_guests_on_host_boot=true

--resume_guests_state_on_host_boot=true

--vnc_enabled=true

--vncproxy_url=http://192.168.1.10:6080

--vnc_console_proxy_url=http://192.168.1.10:6080

# network specific settings

--network_manager=nova.network.manager.FlatDHCPManager

--public_interface=eth0

--flat_interface=eth1

--flat_network_bridge=br100

--fixed_range=192.168.22.128/27

--floating_range=192.168.1.128/27

--network_size=32

--flat_network_dhcp_start=192.168.22.128

--flat_injected=True

--force_dhcp_release=True

--iscsi_helper=tgtadm

--connection_type=libvirt

--root_helper=sudo nova-rootwrap

--network_host=192.168.1.10

--fixed_ip_disassociate_timeout=10

Then euca-run-instances works and I end up with some running VMs.

***However***:

If I do euca-get-console-output on any instance running on the new compute machine I get

....

[ 0.931624] Freeing unused kernel memory: 900k freed

[ 0.933522] Write protecting the kernel read-only data: 12288k

[ 0.942549] Freeing unused kernel memory: 1956k freed

[ 0.946482] Freeing unused kernel memory: 1364k freed

Loading, please wait...

Begin: Loading essential drivers ... done.

[ 0.960394] udevd[79]: starting version 173

Begin: Running /scripts/init-premount ... done.

Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.

[ 1.060068] usb 1-1: new full speed USB device number 2 using uhci_hcd

[ 1.092455] FDC 0 is a S82078B

Begin: Running /scripts/local-premount ... [ 1.120117] Refined TSC clocksource calibration: 3191.993 MHz.

done.

[ 1.228048] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null)

Begin: Running /scripts/local-bottom ... done.

done.

Begin: Running /scripts/init-bottom ... done.

lxcmount stop/pre-start, process 261

cloud-init start-local running: Thu, 10 May 2012 13:54:56 +0000. up 4.97 seconds

no instance data found in start-local

cloud-init-nonet waiting 120 seconds for a network device.