title: NoValidHost: No valid host was found. There are not enough hosts available
body: <p>When I create the instance in the dashboard, I get error:</p>
  
  <blockquote>
    <p>No valid host was found. There are not enough hosts available.</p>
  </blockquote>
  
  <p>In the <code>/var/log/nova/nova-conductor.log</code> file, there is the log:</p>
  
  <pre><code>2017-08-05 00:22:29.046 3834 WARNING nova.scheduler.utils [req-89c159c7-b40a-43eb-8f0d-9306eb73e83a 2a5fa182fb1b459980db09cd1572850e 0d5998f2f7ec4c4892a32e06bafb19df - - -] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available.
  Traceback (most recent call last):
  
    File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 199, in inner
      return func(*args, **kwargs)
  
    File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 104, in select_destinations
      dests = self.driver.select_destinations(ctxt, spec_obj)
  
    File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 74, in select_destinations
      raise exception.NoValidHost(reason=reason)
  
  NoValidHost: No valid host was found. There are not enough hosts available.
  
  2017-08-05 00:22:29.048 3834 WARNING nova.scheduler.utils [req-89c159c7-b40a-43eb-8f0d-9306eb73e83a 2a5fa182fb1b459980db09cd1572850e 0d5998f2f7ec4c4892a32e06bafb19df - - -] [instance: 2011e343-c8fc-4ed0-8148-b0d2b5ba37c3] Setting instance to ERROR state.
  2017-08-05 00:22:30.785 3834 WARNING oslo_config.cfg [req-89c159c7-b40a-43eb-8f0d-9306eb73e83a 2a5fa182fb1b459980db09cd1572850e 0d5998f2f7ec4c4892a32e06bafb19df - - -] Option "auth_plugin" from group "neutron" is deprecated. Use option "auth_type" from group "neutron".
  </code></pre>
  
  <hr>
  
  <p>And I searched the SO, find a related post:<a href="https://stackoverflow.com/questions/35184879/openstack-devstack-cant-create-instance-there-are-not-enough-hosts-available">Openstack-Devstack: Can&#39;t create instance, There are not enough hosts available</a></p>
  
  <p>I checked the <code>free_ram_mb</code> in mysql:</p>
  
  <pre><code>MariaDB [nova]&gt; select * from compute_nodes \G;
  *************************** 1. row ***************************
             created_at: 2017-08-04 12:44:26
             updated_at: 2017-08-04 13:51:35
             deleted_at: NULL
                     id: 4
             service_id: NULL
                  vcpus: 8
              memory_mb: 7808
               local_gb: 19
             vcpus_used: 0
         memory_mb_used: 512
          local_gb_used: 0
        hypervisor_type: QEMU
     hypervisor_version: 1005003
               cpu_info: {"vendor": "Intel", "model": "Broadwell", "arch": "x86_64", "features": ["smap", "avx", "clflush", "sep", "rtm", "vme", "invpcid", "msr", "fsgsbase", "xsave", "pge", "erms", "hle", "cmov", "tsc", "smep", "pcid", "pat", "lm", "abm", "adx", "3dnowprefetch", "nx", "fxsr", "syscall", "sse4.1", "pae", "sse4.2", "pclmuldq", "fma", "tsc-deadline", "mmx", "osxsave", "cx8", "mce", "de", "rdtscp", "ht", "pse", "lahf_lm", "rdseed", "popcnt", "mca", "pdpe1gb", "apic", "sse", "f16c", "ds", "invtsc", "pni", "aes", "avx2", "sse2", "ss", "hypervisor", "bmi1", "bmi2", "ssse3", "fpu", "cx16", "pse36", "mtrr", "movbe", "rdrand", "x2apic"], "topology": {"cores": 2, "cells": 1, "threads": 1, "sockets": 4}}
   disk_available_least: 18
            free_ram_mb: 7296
           free_disk_gb: 19
       current_workload: 0
            running_vms: 0
    hypervisor_hostname: ha-node1
                deleted: 0
                host_ip: 192.168.8.101
    supported_instances: [["i686", "qemu", "hvm"], ["x86_64", "qemu", "hvm"]]
              pci_stats: {"nova_object.version": "1.1", "nova_object.changes": ["objects"], "nova_object.name": "PciDevicePoolList", "nova_object.data": {"objects": []}, "nova_object.namespace": "nova"}
                metrics: []
        extra_resources: NULL
                  stats: {}
          numa_topology: NULL
                   host: ha-node1
   ram_allocation_ratio: 3
   cpu_allocation_ratio: 16
                   uuid: 9113940b-7ec9-462d-af06-6988dbb6b6cf
  disk_allocation_ratio: 1
  *************************** 2. row ***************************
             created_at: 2017-08-04 12:44:34
             updated_at: 2017-08-04 13:50:47
             deleted_at: NULL
                     id: 6
             service_id: NULL
                  vcpus: 8
              memory_mb: 7808
               local_gb: 19
             vcpus_used: 0
         memory_mb_used: 512
          local_gb_used: 0
        hypervisor_type: QEMU
     hypervisor_version: 1005003
               cpu_info: {"vendor": "Intel", "model": "Broadwell", "arch": "x86_64", "features": ["smap", "avx", "clflush", "sep", "rtm", "vme", "invpcid", "msr", "fsgsbase", "xsave", "pge", "erms", "hle", "cmov", "tsc", "smep", "pcid", "pat", "lm", "abm", "adx", "3dnowprefetch", "nx", "fxsr", "syscall", "sse4.1", "pae", "sse4.2", "pclmuldq", "fma", "tsc-deadline", "mmx", "osxsave", "cx8", "mce", "de", "rdtscp", "ht", "pse", "lahf_lm", "rdseed", "popcnt", "mca", "pdpe1gb", "apic", "sse", "f16c", "ds", "invtsc", "pni", "aes", "avx2", "sse2", "ss", "hypervisor", "bmi1", "bmi2", "ssse3", "fpu", "cx16", "pse36", "mtrr", "movbe", "rdrand", "x2apic"], "topology": {"cores": 2, "cells": 1, "threads": 1, "sockets": 4}}
   disk_available_least: 18
            free_ram_mb: 7296
           free_disk_gb: 19
       current_workload: 0
            running_vms: 0
    hypervisor_hostname: ha-node2
                deleted: 0
                host_ip: 192.168.8.102
    supported_instances: [["i686", "qemu", "hvm"], ["x86_64", "qemu", "hvm"]]
              pci_stats: {"nova_object.version": "1.1", "nova_object.changes": ["objects"], "nova_object.name": "PciDevicePoolList", "nova_object.data": {"objects": []}, "nova_object.namespace": "nova"}
                metrics: []
        extra_resources: NULL
                  stats: {}
          numa_topology: NULL
                   host: ha-node2
   ram_allocation_ratio: 3
   cpu_allocation_ratio: 16
                   uuid: 32b574df-52ac-43dc-87f8-353350449076
  disk_allocation_ratio: 1
  2 rows in set (0.00 sec)
  </code></pre>
  
  <p>You see the <code>free_ram_mb: 7296</code>, I just want to create a <code>512</code>mb VM, but failed.</p>
  
  <hr>
  
  <p><strong>EDIT-1</strong></p>
  
  <p>The nova services all are up:</p>
  
  <pre><code>[root@ha-node1 ~]# nova service-list
  +----+------------------+----------+----------+---------+-------+----------------------------+-----------------+
  | Id | Binary           | Host     | Zone     | Status  | State | Updated_at                 | Disabled Reason |
  +----+------------------+----------+----------+---------+-------+----------------------------+-----------------+
  | 2  | nova-consoleauth | ha-node3 | internal | enabled | up    | 2017-08-05T14:20:25.000000 | -               |
  | 5  | nova-conductor   | ha-node3 | internal | enabled | up    | 2017-08-05T14:20:29.000000 | -               |
  | 7  | nova-cert        | ha-node3 | internal | enabled | up    | 2017-08-05T14:20:23.000000 | -               |
  | 15 | nova-scheduler   | ha-node3 | internal | enabled | up    | 2017-08-05T14:20:20.000000 | -               |
  | 22 | nova-cert        | ha-node1 | internal | enabled | up    | 2017-08-05T14:20:26.000000 | -               |
  | 29 | nova-conductor   | ha-node1 | internal | enabled | up    | 2017-08-05T14:20:22.000000 | -               |
  | 32 | nova-consoleauth | ha-node1 | internal | enabled | up    | 2017-08-05T14:20:29.000000 | -               |
  | 33 | nova-consoleauth | ha-node2 | internal | enabled | up    | 2017-08-05T14:20:29.000000 | -               |
  | 36 | nova-scheduler   | ha-node1 | internal | enabled | up    | 2017-08-05T14:20:30.000000 | -               |
  | 40 | nova-conductor   | ha-node2 | internal | enabled | up    | 2017-08-05T14:20:26.000000 | -               |
  | 44 | nova-cert        | ha-node2 | internal | enabled | up    | 2017-08-05T14:20:27.000000 | -               |
  | 46 | nova-scheduler   | ha-node2 | internal | enabled | up    | 2017-08-05T14:20:28.000000 | -               |
  | 49 | nova-compute     | ha-node2 | nova     | enabled | up    | 2017-08-05T14:19:35.000000 | -               |
  | 53 | nova-compute     | ha-node1 | nova     | enabled | up    | 2017-08-05T14:20:05.000000 | -               |
  +----+------------------+----------+----------+---------+-------+----------------------------+-----------------+
  </code></pre>
  
  <p>The nova list:</p>
  
  <pre><code>[root@ha-node1 ~]# nova list 
  +--------------------------------------+------+--------+------------+-------------+----------+
  | ID                                   | Name | Status | Task State | Power State | Networks |
  +--------------------------------------+------+--------+------------+-------------+----------+
  | 20193e58-2c5b-44c6-a98f-a44e2001934f | vm1  | ERROR  | -          | NOSTATE     |          |
  </code></pre>
  
  <p>And the nova show instance:</p>
  
  <pre><code>[root@ha-node1 ~]# nova show 20193e58-2c5b-44c6-a98f-a44e2001934f
  +--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  | Property                             | Value                                                                                                                                                                                                      |
  +--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  | OS-DCF:diskConfig                    | AUTO                                                                                                                                                                                                       |
  | OS-EXT-AZ:availability_zone          | nova                                                                                                                                                                                                       |
  | OS-EXT-SRV-ATTR:host                 | -                                                                                                                                                                                                          |
  | OS-EXT-SRV-ATTR:hostname             | vm1                                                                                                                                                                                                        |
  | OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                                                                                                                                                                                          |
  | OS-EXT-SRV-ATTR:instance_name        | instance-00000003                                                                                                                                                                                          |
  | OS-EXT-SRV-ATTR:kernel_id            |                                                                                                                                                                                                            |
  | OS-EXT-SRV-ATTR:launch_index         | 0                                                                                                                                                                                                          |
  | OS-EXT-SRV-ATTR:ramdisk_id           |                                                                                                                                                                                                            |
  | OS-EXT-SRV-ATTR:reservation_id       | r-jct8kkcq                                                                                                                                                                                                 |
  | OS-EXT-SRV-ATTR:root_device_name     | /dev/vda                                                                                                                                                                                                   |
  | OS-EXT-SRV-ATTR:user_data            | -                                                                                                                                                                                                          |
  | OS-EXT-STS:power_state               | 0                                                                                                                                                                                                          |
  | OS-EXT-STS:task_state                | -                                                                                                                                                                                                          |
  | OS-EXT-STS:vm_state                  | error                                                                                                                                                                                                      |
  | OS-SRV-USG:launched_at               | -                                                                                                                                                                                                          |
  | OS-SRV-USG:terminated_at             | -                                                                                                                                                                                                          |
  | accessIPv4                           |                                                                                                                                                                                                            |
  | accessIPv6                           |                                                                                                                                                                                                            |
  | config_drive                         |                                                                                                                                                                                                            |
  | created                              | 2017-08-05T14:17:54Z                                                                                                                                                                                       |
  | description                          | vm1                                                                                                                                                                                                        |
  | fault                                | {"message": "No valid host was found. There are not enough hosts available.", "code": 500, "details": "  File \"/usr/lib/python2.7/site-packages/nova/conductor/manager.py\", line 496, in build_instances |
  |                                      |     context, request_spec, filter_properties)                                                                                                                                                              |
  |                                      |   File \"/usr/lib/python2.7/site-packages/nova/conductor/manager.py\", line 567, in _schedule_instances                                                                                                    |
  |                                      |     hosts = self.scheduler_client.select_destinations(context, spec_obj)                                                                                                                                   |
  |                                      |   File \"/usr/lib/python2.7/site-packages/nova/scheduler/utils.py\", line 370, in wrapped                                                                                                                  |
  |                                      |     return func(*args, **kwargs)                                                                                                                                                                           |
  |                                      |   File \"/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py\", line 51, in select_destinations                                                                                             |
  |                                      |     return self.queryclient.select_destinations(context, spec_obj)                                                                                                                                         |
  |                                      |   File \"/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py\", line 37, in __run_method                                                                                                    |
  |                                      |     return getattr(self.instance, __name)(*args, **kwargs)                                                                                                                                                 |
  |                                      |   File \"/usr/lib/python2.7/site-packages/nova/scheduler/client/query.py\", line 32, in select_destinations                                                                                                |
  |                                      |     return self.scheduler_rpcapi.select_destinations(context, spec_obj)                                                                                                                                    |
  |                                      |   File \"/usr/lib/python2.7/site-packages/nova/scheduler/rpcapi.py\", line 126, in select_destinations                                                                                                     |
  |                                      |     return cctxt.call(ctxt, 'select_destinations', **msg_args)                                                                                                                                             |
  |                                      |   File \"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py\", line 169, in call                                                                                                                |
  |                                      |     retry=self.retry)                                                                                                                                                                                      |
  |                                      |   File \"/usr/lib/python2.7/site-packages/oslo_messaging/transport.py\", line 97, in _send                                                                                                                 |
  |                                      |     timeout=timeout, retry=retry)                                                                                                                                                                          |
  |                                      |   File \"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py\", line 464, in send                                                                                                       |
  |                                      |     retry=retry)                                                                                                                                                                                           |
  |                                      |   File \"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py\", line 455, in _send                                                                                                      |
  |                                      |     raise result                                                                                                                                                                                           |
  |                                      | ", "created": "2017-08-05T14:18:14Z"}                                                                                                                                                                      |
  | flavor                               | m1.tiny (1)                                                                                                                                                                                                |
  | hostId                               |                                                                                                                                                                                                            |
  | host_status                          |                                                                                                                                                                                                            |
  | id                                   | 20193e58-2c5b-44c6-a98f-a44e2001934f                                                                                                                                                                       |
  | image                                | cirros-0.3.4-x86_64 (202778cd-6b32-4486-9444-c167089d9082)                                                                                                                                                 |
  | key_name                             | -                                                                                                                                                                                                          |
  | locked                               | False                                                                                                                                                                                                      |
  | metadata                             | {}                                                                                                                                                                                                         |
  | name                                 | vm1                                                                                                                                                                                                        |
  | os-extended-volumes:volumes_attached | []                                                                                                                                                                                                         |
  | status                               | ERROR                                                                                                                                                                                                      |
  | tags                                 | []                                                                                                                                                                                                         |
  | tenant_id                            | 0d5998f2f7ec4c4892a32e06bafb19df                                                                                                                                                                           |
  | updated                              | 2017-08-05T14:18:16Z                                                                                                                                                                                       |
  | user_id                              | 2a5fa182fb1b459980db09cd1572850e                                                                                                                                                                           |
  +--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  </code></pre>
  
  <hr>
  
  <p><strong>EDIT-2</strong></p>
  
  <p>The <code>nova-compute.log</code> in the <code>/var/log/nova/</code> useful information:</p>
  
  <pre><code>......
  2017-08-05 22:17:42.669 103174 INFO nova.compute.resource_tracker [req-60a062ce-4b3d-4cb7-863e-2f9bba0bc6ec - - - - -] Compute_service record updated for ha-node1:ha-node1
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [req-7dded91f-7497-4d20-ba89-69f867a2a8fb 2a5fa182fb1b459980db09cd1572850e 0d5998f2f7ec4c4892a32e06bafb19df - - -] [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f] Instance failed to spawn
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f] Traceback (most recent call last):
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2078, in _build_resources
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f]     yield resources
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1920, in _build_and_run_instance
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f]     block_device_info=block_device_info)
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2584, in spawn
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f]     admin_pass=admin_password)
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2959, in _create_image
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f]     fileutils.ensure_tree(libvirt_utils.get_instance_path(instance))
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f]   File "/usr/lib/python2.7/site-packages/oslo_utils/fileutils.py", line 40, in ensure_tree
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f]     os.makedirs(path, mode)
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f]   File "/usr/lib64/python2.7/os.py", line 157, in makedirs
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f]     mkdir(name, mode)
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f] OSError: [Errno 13] Permission denied: '/var/lib/nova/instances/20193e58-2c5b-44c6-a98f-a44e2001934f'
  2017-08-05 22:18:03.357 103174 ERROR nova.compute.manager [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f]
  2017-08-05 22:18:11.563 103174 INFO nova.compute.manager [req-7dded91f-7497-4d20-ba89-69f867a2a8fb 2a5fa182fb1b459980db09cd1572850e 0d5998f2f7ec4c4892a32e06bafb19df - - -] [instance: 20193e58-2c5b-44c6-a98f-a44e2001934f] Terminating instance
  ....
  </code></pre>