Nutanix – Turn-off IPMI VLAN Tagging using IPMITool

Yesterday I faced an issue which took more 2-3 hour to get sorted. I had old Nutanix Block on which I need to do the foundation when I started the foundation using Nutanix Foundation VM and got error foundation process failed with error “ Unable to set IPMI IPs)

  • I tried to set IPMI IP on the Nutanix Nodes manually, but strange thing even after setting up IPMI IPs I was unable to ping the IPMI interface. I tried on Shared/ dedicated IPMI Port but failed.
  • The hardware I was using was old Nutanix Block from G4 Generation, and there was no other options in IPMI Menu in System Setup to modify except IP Address
  • I tried to login Hypervisor , but failed as I don’t have IP root password.

Moving forward, I got an idea make one bootable using download IPMItool from SuperMicro Website, boot the nodes from USB Drive.

  • When I run the IPMICFG tool and get the current VLAN Status, VLAN ID was ON on the IPMI interface. I turned off the VLAN ID.

IPMICFG -vlan off

Incase if you have access on the hypervisor IPMItool can be used to trun-off the VLAN ID, using below command to turn off VLAN Id from ESXi Host.

/ipmitool lan set 1 vlan ID off

Once the VLAN off, I am able to access IPMI ports and proceed with foundation.

IPMICFG Can be download from below link

https://www.supermicro.com/SwDownload/SwSelect_Free.aspx?cat=IPMI

AHV Networking Basics

AHV use Open VSwitch(OVS) to connect the CVM, Hypervisor and gest Virtual machines to each other and to physical network.

Open VSwitch:

Open VSwitch is an opensource software switch implementation in the Linux Kernel. OVS acts like a Layer-2 switch which means its maintain the MAC Address table. Host, VMs connect to Virtual Ports on the switch.OVS support common switch features like VLAN Tagging, Load Balancing and link Aggregation Control Protocol (LACP). Each AHV Host server maintain and OVS instance and all OVS instance combine to form a single logical switch.

Bridges:

Bridges act as virtual switch to manage traffic between physical and virtual network interfaces. By default AHV configuration includes an OVS called Br0 / Br0-up and a native Linux Bridge call Virbr0. The virbr0 Linux bridge is responsible for management and storage communication between CVM and AHV host. All the other traffic (storage, host, VM) flows through br0 OVS bridge.

Ports:

Ports are logical entity created in bridge that represent connectivity to virtual switch.

Bonds:

Bonded ports aggregate the physical interfaces on the AHV host. By default, the system creates a bond named br0-up /Br0 having all physical interfaces.

OVS bonds supports multiple load-balancing modes, including active-backup, balance-slb, and balance-tcp.

Unable to login to VSphere 6.7 Appliance Management UI

Today I have faced the strange issue, I tried to add new host to vcenter but unable to add to check the VCenter services I tried to login to Vcenter appliance Management UI but unable to login.

I tried to reset the Vcenter Appliance root passowrd but didn’t work. After troubleshoting I found the solution. Applmgmt Services was not running which was not allowing me login to management Console.

So Check the services status from Shell , open VCenter Appliance Console / SSH.

Access à Shell

#service-control –status

#service-control –start applmgmt

Once Services started , I am able to login to console & able to add ESXi host to VCenter.

How to Deploy VMWare VCenter Appliance in VMware WorkStation

As we all know VMware WorkStation is a great solution for testing /home lab activities. Lots of guys ask me how to deploy VMware VCenter Appliance deploy in VMware Workstation as using VCenter Installer it looks for ESXi host to deploy the VCenter Appliance. So below is the procedure you can follow to deploy VCenter Server on VMware Workstation.

  • Download the VCenter Appliance ISO
  • Extract / Mount ISO
  • Browse to folder VSA and double click on OVA file, wizard will be started to import the OVA to VMware Workstation.

Accept the License Agreement and click Next

Specify the Virtual Machine name & location and click Next

Specify the Deployment Type and click Next

  • Specify the Network Configuration
  • Host network Family: IPV4 or IPV6
  • Host Network mode : Static / Dynamic
  • Host IP Address:
  • Host network Profex / Subnet
  • Default gateway
  • DNS Server IP
  • Virtual machine DNS Name

Once provide the information Click next.

Once VCenter Appliance will be deployed. Logged into Virtual machines

Set the VCenter Appliance IP and access the Virtual Machines Management using any explorer

https:// Virtual Appliance IP:5480

Now the Stage 2, will be started. Click Setup to proceed.

Click Next start the Stage 2

Provide the NTP information, Enable SSH Access incase if you want to configure VCenter Sever HA.

As I am planning to configure VCenter HA, hence enabling SSH Access. Click Next to continue.

Select the desire SSO Configuration, as I am going to setup the new SSO Domain. So I am providing the SSO domain name & Password for the Administrator. Click Next to continue.

Select the CEP option you desire and click next.

Confirm the Summary information and click finish to start the STAGE 2 Configuration. Please note that once you click Next you can’t stop the process.

Process will take upto 10-15 mins to finish the deployment.

Below will the URLs to access the appliance.

Appliance Management:

https://appliance_ip/hostname:5480

VCenter

https://appliance_ip/hostname:443

How to Configure VCenter Server Appliance HA

VCenter HA is active – Passive solution, A VCenter Appliance HA Cluster consists of three VCenter Instances, 1st instance used as active and cloned 2 times as Passive & witness nodes.

VCenter HA Requirements:

  • Minimum of three ESXi 6.0 or Higher Hosts
  • Each VCenter will run on different hosts

VCenter Server 6.5 and higher

  • Deploy Small Size (4VCP & 16GB RAM) or higher is required. Not recommend to use tiny in production Environments

Network Connectivity

  • Less than 10ms. Latency between Active, Passive and witness nodes
  • VCenter HA network must be on different SUBNET than the management network. L2 network is recommended.

Setting up VCenter Appliance HA:

Select VCenter à Configuration à VCenter HA


Select “Set Up VCenter HA” to continue the VCenter HA Configuration


Specify the VCenter HA Network, Additional NIC (VCenter HA Network) will be added in active VCenter.


Click Edit and Specify the deployment host, Storage and network for Passive Network.


Specify the Host Click Next


Specify the Datastore where you want to store the passive VCenter node.


Specify the Network for passive Node


Click Finish. Repeat the same for Witness VM.


Click Next to configure VCenter Nodes HA IPs.


Click Finish to Start the Process. Whole process can take upto 15-20 min depends on the VM Clone copy process.


VCenter HA has been configured. You can see two new VM Named with passive & Witness has been created.

Manual Failover


Click on Initiate Failover to start the manual failover.


Click Initiate Failover, to confirm the failover. Select force option if you want to initate the failover immediately.

Destroy VCener HA


Click Remove VCenter HA to destroy the Center HA configuration

How to Upgrade VMWare VCenter Appliance 6.6 to 7.0

As all we know VMWare has release the VSphere 7.0. In this article I will show how to upgrade VCenter Appliance 6.7 to 7.0.

Download VCenter Server Appliance from VMware Website. And mount the ISO. Browse to vcsa-ui-installer on Mounted Media and run installer.exe

Note: Before starting the upgrade, make sure DNS record and NTP is working fine. VCenter Appliance upgrade process will deploy the new VCenter appliance and copy the configuration and data from old appliance to newly deployed appliance. This means temporary you should the enough resources to accommodate the new VM.

Click Upgrade to start the Wizard.

Click Next to continue

Accept the license agreement and click Next

Connect the VCenter appliance

Provide the ESXi Server where you want to deploy the new VCenter VM.

Provide the New VCenter VM name & provide the appliance Root Password.

Select deployment size and click next.

Select the datastore, incase if you want to save the disk as thin format select enable thin disk mode.

Provide the network information and temporary IP.

Review the configuration and click Finish to start the deployment.

In this Stage VM is already deployed, once deployment finished you can continue to start the Stage 2. In stage 2, VM will copy the data from old appliance and configure the new appliance. Click Continue

Click the next to start the wizard.

Select the upgrade data and click next.

Select the desire CEP option and click next.

Review the configuration and click Finish to start the process.

Accept the Warning, saying source VM will be shut down when new VM get the network configuration.

Whole process takes approx. 15-20 Min to finished.

Upgrade finished, Click Close to close the installation wizard.

VCenter is upgraded. You can access the VCenter appliance

https://vsa.demo.local:443

First thing to be noticed, HTML5 is only available GUI going forward.

How to Deploy VMware VCenter Appliance

vCenter Server is the centralized management utility for VMware, and is used to manage virtual machines, multiple ESXi hosts, and all dependent components from a single centralized location. VMware vMotion and svMotion require the use of vCenter and ESXi hosts.

In this article, I will show Step by Step instruction how to deploy VMWare VCenter Server Appliance.

VCenter Server Appliance Requirements:

Resource Requirement
Disk storage on the host machine Embedded Platform Services Controller:

  • Tiny: 120GB
  • Small: 150GB
  • Medium: 300GB
  • Large: 450GB

vCenter Server Appliance with an External Platform Services Controller:

  • Tiny: 86GB
  • Small: 108GB
  • Medium: 220GB
  • Large: 280GB

External Platform Services Controller Appliance:

  • Tiny: 30GB
  • Small: 30GB
  • Medium: 30GB
  • Large: 30GB
Memory in the vCenter Server Appliance Platform Services Controller Only: 2GB Ram

All components on one Appliance.

  • Tiny: 8GB RAM
  • Small: 16GB RAM
  • Medium: 24GB RAM
  • Large: 32GB RAM
CPUs in the vCenter Server Appliance Platform Services Controller Only: 2 CPUs

All components on one Appliance.

  • Tiny: 2 CPUs
  • Small: 4 CPUs
  • Medium: 8 CPUs
  • Large: 16 CPUs

Notes:

  • Tiny Environment (up to 10 Hosts, 100 Virtual Machines)
  • Small Environment (up to 100 Hosts, 1,000 Virtual Machines)
  • Medium Environment (up to 400 Hosts, 4,000 Virtual Machines)
  • Large Environment (up to 1,000 Hosts, 10,000 Virtual Machines)

Download VCenter Server Appliance from VMware Website. And mount the ISO. Browse to vcsa-ui-installer on Mounted Media and run installer.exe

Note: Before starting the installed, make sure to create the DNS A-Record for the VCenter Server along with reverse pointer for the same.

Click Install to start the Installation Wizward.

Click Next start the Appliance Deploy.

Accept License Agreement and click next

Select the Desire deployment type and click Next. I am proceeding with Embedded Platform in this article.

Provide the ESXI or VCenter Server information where you want to deploy the VCenter Appliance. And check ok.

Accept the ESX Server Certificate Warning.

ESXI is connected Click Next.

Based on the requirements select the deployment size, and storage size.

Select the datastore where you want to save the Appliance. Incase if you want to deploy the appliance in Thin Disk Mode, select the option “Enable Thin Disk Mode”

Provide the Network information, like VM IP hostname etc. Make sure before proceed next create the DNS Entries.

Click Finish to start the Appliance VM deployment.

Once VM Deployment finished, Stage 1 of VCenter Appliance got finished. Click Continue to start the Stage 2 of deployment.

Click Next start the Stage 2

Provide the NTP information, Enable SSH Access incase if you want to configure VCenter Sever HA.

As I am planning to configure VCenter HA, hence enabling SSH Access. Click Next to continue.

Select the desire SSO Configuration, as I am going to setup the new SSO Domain. So I am providing the SSO domain name & Password for the Administrator. Click Next to continue.

Select the CEP option you desire and click next.

Confirm the Summary information and click finish to start the STAGE 2 Configuration. Please note that once you click Next you can’t stop the process.

Process will take upto 10-15 mins to finish the deployment.

Below will the URLs to access the appliance.

Appliance Management:

https://appliance_ip/hostname:5480

VCenter

https://appliance_ip/hostname:443

How to Fix the problem with Root password to VMware vCenter (VCSA 6.7 console)

I tried to login to VCenter Appliance Web interface after very long time, and notice unable to login on appliance and below error.

The quick fix for the error is log in to Appliance Console/SSH and goes to Shell and run the passwd command and provide the new password, then repeat the new password. Once password updated. You can login to web console.

    Upgrading ESXi Host using Offline-Bundle

    In this article I will show how to upgrade VMware ESXi version using offline-Bundle.

    • Download the VMware Esxi Offline Bundle
    • Upload the Offline-Bundle to ESXi datastore
    • Place the ESXi host in Maintenance mode
    • SSH to ESXi Host using putty
    • Run the upgrade command , replace the DATASTORE-UUID with the UUID of your datastore where you have uploaded the Offline Bundle

    esxcli software vib update -d /vmfs/volumes/<DATASTORE-UUID>/update-from-esxi6.5-xxx.zip

    • once done, reboot the host using below command
    • Reboot
    • Exit maintenance mode

    Upgrade Compatibility Checks

    https://www.vmware.com/resources/compatibility/sim/interop_matrix.php#upgrade&solution=1

    VMware Hardware Compatibility List

    https://www.vmware.com/resources/compatibility/search.php

    Exchange Database not getting listed in GUI

    I faced the strange issues with one of my customer, where Exchange Database was not getting list in GUI, as well as when I try to list the Databases using Exchange Management Shell I got the error saying unable to find “Mailbox XXXXX “ DB.

    After seen this error, I opened ADSI Edit and found someone didn’t decommission the old Exchange properly. Old Exchange Sever was removed and Old Databases still exists in Active Directory Schema. After identify this , I just clean-up the old Database entries and do a refresh on ECP all databases got list immediately.

    • Login to domain controller with administrative account.
    • Navigate to Start –> Run –> ADSIEdit.msc and hit enter
    • Click on Action –> Connect to –> Select Configuration under “Select a well known naming Context:”

    Navigate to CN=Configuration, DC=DOMAINNAME, DC=COM

    Replace DC=DOMAINNAME, DC=COM with your domain name

    Expand the CN=Services –> CN=Microsoft Exchange

    After Removing the all old DBs, my current DBs started getting listed in Exchange GUI & Management Shell.

    Note: I would recommend always use proper methods to old exchange servers, use adsi edit only incase Exchange server is no more in the environment or its got crashed. Improper use of ADSI edit may harm the active directory schema, so be very careful while modifying/deleting any values /keys.