Archive

Archive for the ‘VMWare’ Category

Create ESXi Host firewall rules using PowerCli

February 13th, 2015 No comments

Recently, I needed to create some custom firewall rules on ESX hosts (lots of them) – for some Syslog servers that were in addition to our existing Syslog servers.

Here was my appraoach (note – Firewall rules will be lost at reboot unless you hack bootbank, or make the changes using custom VIBs)
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2007381

http://cormachogan.com/2014/03/28/adding-bespoke-firewall-rules-to-esxi/

The code has dependecnies on Posh-SSH (freely available, excellent automation tool)

# PowerCLI Script for enabling SSH, setting firewall and disable SSH alert in vCenter
# multiple dependencies 
# https://github.com/darkoperator/Posh-SSH 
# PowerCli


#Requires -Modules ActiveDirectory
#Requires -Version 4
#Requires -Modules Posh-SSH


# Sample update
import-module posh-ssh
add-pssnapin vm*
$vmhostname = read-host "Please provide target ESX hostname"
Connect-VIServer <viservname>
$Credential = Get-Credential -Message "Please provide root user password for $vmhostname" -UserName "root"

# enable SSH on ESX host (function below)
Set-SshOnVmHost -VMHostName $vmhostname -Status On

# Create firewall rule on ESX host
new-FireWallRuleonVmHost -VMHostName $vmhostname -direction outbound -protocol tcp -porttype dst -serviceID myServicename -port 1234 -enabled false -required false

# Display the change
$esxcli = Get-EsxCli -VMHost $vmhostname
$esxcli.network.firewall.ruleset.list()

# Disable SSH on the host
Set-SshOnVmHost -VMHostName $vmhostname -Status Off


function Set-SshOnVmHost
{
    Param(
        [String]
        $VMHostName,
        [ValidateSet('On','Off')]
        [String]
        $Status
        )

    	write-host "Configuring SSH on host: $($vmHost.Name) to $Status" -fore Yellow
        if((Get-VMHostService -VMHost $vmhostname | where {$_.Key -eq "TSM-SSH"}).Policy -ne "$Status"){
            Write-Host "Setting SSH service policy to automatic $status $($vmHost.Name)"
            Get-VMHostService -VMHost $vmhostname | where { $_.key -eq "TSM-SSH" } | Set-VMHostService -Policy "$Status" -Confirm:$false -ea 1 | Out-null
	    }
    
        $vmhost = get-vmhost $vmhostname
      
        $esxcli = Get-EsxCli -VMHost $vmhost
        if($Status -eq 'On'){
	        if((Get-VMHostService -VMHost $vmhost | where {$_.Key -eq "TSM-SSH"}).Running -ne $true){
                Write-Host "Starting SSH service on $($vmHost.Name)"
                Start-VMHostService -confirm:$false  -HostService (Get-VMHost $vmHost | Get-VMHostService | Where { $_.Key -eq "TSM-SSH"}) | Out-null
                $ip = (Get-WmiObject -class win32_NetworkAdapterConfiguration -Filter 'ipenabled = "true"').ipaddress[0]
	            if($esxcli -ne $null){
                    if($ip.count -ne 1){
                        $ip = read-host "Please provide the IP address that should be to access this host via SSH"
                    }
                    try{
                        test-ipaddress $ip
                        if(($esxcli.network.firewall.ruleset.allowedip.list("sshServer") | select AllowedIPAddresses).AllowedIPAddresses -eq "All"){
                            Write-Host "Changing the sshServer firewall configuration"        
                            $esxcli.network.firewall.ruleset.set($false, $true, "sshServer")
                             if(($esxcli.network.firewall.ruleset.allowedip.list("sshServer") | select AllowedIPAddresses).AllowedIPAddresses -notmatch "$ip"){
                                $esxcli.network.firewall.ruleset.allowedip.add("$ip", "sshServer")
                            }
                            $esxcli.network.firewall.refresh()
                        }
                    }
                    catch {
                        throw "Unable to configure IP restirctions on Firewall when enabling SSH $_"		        
                    }
		        }
            }
        }
        else{
	        if((Get-VMHostService -VMHost $vmhost | where {$_.Key -eq "TSM-SSH"}).Running -ne $false){
                Write-Host "Stopping SSH service on $($vmHost.Name)"
                Stop-VMHostService -confirm:$false -HostService (Get-VMHost $vmHost | Get-VMHostService | Where { $_.Key -eq "TSM-SSH"}) | Out-null
                write-host "Set Firewall rule too allow all IPs for SSH, but disable the service"
		        if(($esxcli.network.firewall.ruleset.allowedip.list("sshServer") | select AllowedIPAddresses).AllowedIPAddresses -ne "All"){
        	        Write-Host "Changing the sshServer firewall configuration"        
        		        $esxcli.network.firewall.ruleset.set($true, $true, "sshServer")
            	        $esxcli.network.firewall.refresh()
                }
            }
        }
            
	    # End Comment
	    if(($vmHost | Get-AdvancedSetting | Where {$_.Name -eq "UserVars.SuppressShellWarning"}).Value -ne "1"){
            Write-Host "Suppress the SSH warning message"
        	    $vmHost | Get-AdvancedSetting | Where {$_.Name -eq "UserVars.SuppressShellWarning"} | Set-AdvancedSetting -Value "1" -Confirm:$false | Out-null
        }
 }

 function Test-IPaddress
{
    [CmdletBinding()]
    Param
    (
        [Parameter(Mandatory=$true,
                   ValueFromPipelineByPropertyName=$true,
                   Position=0)]
        [ValidateScript({$_ -match [IPAddress]$_ })]  
        [string]
        $IPAddress
    )

        [ipaddress]$IPAddress
}


# function for creation of a new Firewall rulle on an ESX host

function new-FireWallRuleonVmHost {
 Param( [Parameter(Mandatory=$true)]
        [String]
        $VMHostName,
        [Parameter(Mandatory=$true)]
        [ValidateSet('inbound','outbound')]
        [String]
        $direction,
        [Parameter(Mandatory=$true)]
        [ValidateSet('tcp','udp')]
        [String]
        $protocol,
        [Parameter(Mandatory=$true)]
        [ValidateSet('dst','src')]
        [String]
        $porttype,
        [Parameter(Mandatory=$true)]
        [String]$serviceID,
        [ValidateRange(0,65536)]
        [int]$port,
        [Parameter(Mandatory=$true)]
        [ValidateSet('true','false')]
        [String]
        $enabled,
        [Parameter(Mandatory=$true)]
        [ValidateSet('true','false')]
        [String]
        $required
        )

    # Create an SSH Session to the target
    $ssh = New-SSHSession -ComputerName $VMHostName -Credential $credential
   
    # Get the SSH Session's ID
    $sessionId = $($ssh.Index)
    
    # Copy original service.xml to .bak
    Invoke-SSHCommand -Command  "cp /etc/vmware/firewall/service.xml /etc/vmware/firewall/service.xml.bak" -Index $sessionId

    # Allow write access over existing service.xml
    Invoke-SSHCommand -Command  "chmod 644 /etc/vmware/firewall/service.xml" -Index $sessionId

    # Change the sticky bit
    Invoke-SSHCommand -Command  "chmod +t /etc/vmware/firewall/service.xml" -Index $sessionId

    # Create a local copy of the service.xml file
    $out = %{Invoke-SSHCommand -Command  "cat /etc/vmware/firewall/service.xml" -Index $sessionId}
    $out.Output > c:\temp\services.xml
    1$xml = $out.Output

    # Sort through IDs to find the highest current value (assuming 
    $ids = @(); $list = $xml.ConfigRoot.service | %{$_.id}  | ?{$_ -match '0'} | %{$ids += [int]$_}
    [string]$id = "`'00" +  (($ids | sort-object -Descending | select-object -First 1) + 1).ToString() + "`'";

    # insert xml for new service custom xml before closing  the closing <`/ConfigRoot`>
    $newXml = (get-content C:\temp\servicestmp.xml | Select-String -Pattern "`<`/ConfigRoot`>" -NotMatch) + @"
    <service id=$id>
        <id>$serviceID</id>
        <rule>
            <direction>$direction</direction>
            <protocol>$protocol</protocol>
            <porttype>$porttype</porttype>
            <port>$port</port>
        </rule>
        <enabled>$enabled</enabled>
        <required>$required</required>
    </service>
</ConfigRoot>
"@ > "C:\temp\services.xml"

    # copy new xml to services.xml on vm host
    Set-SCPFile -ComputerName $VMHostName -Credential $credential -LocalFile C:\temp\services.xml -RemoteFile /etc/vmware/firewall/service.xml
    
     # Prevet write access over service.xml
    Invoke-SSHCommand -Command  "chmod 444 /etc/vmware/firewall/service.xml" -Index $sessionId

    # Change the sticky bit
    Invoke-SSHCommand -Command  "chmod -t /etc/vmware/firewall/service.xml" -Index $sessionId
    
    # Refresh Firewall Rules on ESX host
    Invoke-SSHCommand -Command  "esxcli network firewall refresh" -Index $sessionId
       
    # remove the SSH Session that you created
    remove-sshSession -index $sessionId

}
Categories: Powershell, VMWare Tags:

vExpert 2013 awardees announced

May 29th, 2013 No comments

VMWare (John Troyer/ Corey Romero) has  announced the list of people who have been selected as vExperts for 2013.

http://blogs.vmware.com/vmtn/2013/05/vexpert-2013-awardees-announced.html

I am thrilled and honored to have made onto the panel for a second year running!

Thanks so much to the guys over at VMWare! I look forward to another year.

Congratulations to all the other nominees – both old and new.

 

For those who don’t know, the vExpert award is not a technical certification, but a recognition for what you’ve done for the community during the preceding year

Quote from the announcement:

We’re pleased to announce the list of vExperts for 2013. Each of these vExperts have demonstrated significant contributions to the community and a willingness to share their expertise with others. We are blown away by the passion and knowledge in this group, a group that is responsible for much of the virtualization evangelism that is taking place in the world — both publicly in books, blogs, online forums, and VMUGs; and privately inside customers and VMware partners. Congratulations to you all!

Categories: VMWare Tags:

VMware Forum London – VMware Forum 2013

March 22nd, 2013 No comments

Registration for VMware forum London is open

VMware Forum London – VMware Forum 2013.

Went to this last year, it was a great event, with some fantastic speakers and decent opportunity to network with other guys in the field.
saw quite a few vExperts and VCDXs there last year too.

Categories: VMWare Tags:

VMWare’s revised education paths.

October 10th, 2012 No comments

Just attended a session at VMWorld on VCAP-DCD and figured I’d jot some notes on the revised VMware certification paths.

VMWare are changing the way they run their educational program.

Existing certs are being renamed
E.g. VCP5 being renamed to VCP5-DV – same qualification. This is because multiple solution tracks are now going to be available – as VMWare is of course expanding their offerings.

Changes to certification paths.
Each path will now have the achievement tracks : Professional, Advanced Professional, Expert.

There will be 5 solution tracks – these being:
Cloud, datacenter virtualised(DCD/DCA), end user computing(View), cloud application platform

This allows for role based certification I.e. engineer, administrator, architect or developers can chose paths that suit their specific requirements.
If you go to my learn.vmware.com you can actually search paths based on the roles specified above.
VCP-DV would be professional level, VCAP-DCA/DCD would be advanced professional and of course VCDx would be expert.

Currently 85000 VCPs and 2000 VCAPs and 100 VCDXs worldwide.

Resources:
As ever, information for each of the exams will be available in BluePrints.
For VCAP exams, there are ui demos on Mylearn.vmware.com
Use the VMWare communities
There are severals study guides and books available
You HAVE to have a lab use VmWare workstation, or buy an Hp Microserver or two for about £150
Attend a course if you can afford it

Exams are schedule via Pearson Vue

If you are at VMWorld – there is a lab sim where you can have a crack at a few simulations – definitely worth the time and effort!

Categories: VMWare, VMWorld Tags:

Europe’s biggest geek gathering – VCDX and vExperts unite

October 9th, 2012 No comments

A quick pic from the party hosted by Steve Herron to honor the VCDXs ad vExperts.

I have seen all the big guns already – Mike Laverick, Duncan Epping etc…

20121009-195620.jpg

Not quite sure how I can justify being in present company – I am no geek compared to these guys…

Categories: VMWare, VMWorld Tags:

Round table for UK customers with Steve Herrod

October 9th, 2012 No comments

Attended a roundtable with heavy hitters from the UK client base and Steve Herrod.

Companies present are:
Betfair
William hill
HSBC
NYSE
BUPA
BNP Paribas
AXA tech
Duetsche Bank
Nomura
Experian
John Lewis
Marks and Spencer
Thomson Rueters

Challenges facing clients.

Red global / UK bank – resilient virtual centres
Integration of products
Licensing modelling
Automation of networking
Active active dcs and IP address conflicts
VMware’s speed of movement after acquisitions
Consolidated views of data centres in geographically dispersed datacenters
Private cloud and cloud solutions providers
Licensing challenges due to offerings from other players in the market
Datacenter sprawl
Understanding VMWare’s strep ateliers going forward
Caching read/write for performance
Bringing together products from different vendors
Moving our VMware offerings from QA dev into out production environment
Compute critical apps
Using technology across security domains
Integration of virtual network solutions from external vendors
ITSM integration – not with the tools, but with the process

Steve says:
Many lab type projects in place at VMWare – Only about 8% of projects actually make it through the filters at VMWare – many projects fail.

Server virtualisation – Steve would give VMware an A
Storage virtualisation – B
Networking – C – despite possibly being the world’s third largest switch port provider bed on number allocated (all virtual of course in this case)

Looks to see VMWare focusing on these. Have been looking very aggressively in the networking space.
Network virtualisation is the next big step forward.
Microsoft is a big challenge, but VMWare accepts there will always be multiple players in the virtualisation market, but the value add should make the difference.
Yes strict server consolidation plays into Microsoft’s hands, but check you factor in automation, downtime costs etc. VMWare is a clear winner.

Outside VMWare, companies are working on the integration of physical and physical networks – see VXLan for VMWare.
Also look to see tighter integration with storage providers (hardware plugins/ storage management from vSphere etc)

Check out labs.vmware.com if you have great ideas for direction for VMWare.
To write web based apps – check out www.cloudfoundry.com

VM customers – things they have done:
95% virtualised for non Prod(M&S) – 75% for Prod
Clusters running on VMWare using VCS.
SDRS has solved storage issues where VMs have been thin provisioned – saved 90TB by enabling thin provisioning
BYOD (bring your own device) Horizon

Discussion comparing hyper v and ESX costs.
Hyper v setup time much longer
Hyper V bundled – but much more expensive when you add system center etc
Non windows VMs?

Categories: VMWare, VMWorld Tags:

Cloud infrastructure suite architecture with Duncan Epping

October 9th, 2012 No comments

Managed to snag a seat on Duncan’s Cloud Infrastructure Suite Architecture group session at VMWorld. Of course Duncan is a one of the biggest influences in VMWare’s direction and of course a real wizard – so real excited to get a spot ( I left my session selections a little late – seeing as I only managed to get a blogger’s pass 3 weeks ago)

I’ll update this as we go along . .

12:30 – game time…

20121009-123105.jpg

We have Michael Webster along for the ride (the only kiwi VCDX #66)

What kind of storage enables your cloud?
Legacy fibre?
Legacy IP (iscsi/NFS)?
All flash / SSD?
Hybrid? – most common

What the people are using:
Very few people using sDRS – varying reasons (mostly limitations like incompatibility with SRM etc. / compatibility with backup vendors due to VMS moving location)
Hybrid storage solutions
Some SSD
About 40% using vCloud Director
Jumbo Frames for lowering CPU requirements
Only 5% using network IO control
Almost everyone on Enterprise or enterprise plus.
VSwitches still rule – low uptake of vDS
Guys using Nexus 1000v to keep networks team happy?

vSphere 5 has removed the 8 host limitation for vmfs access.

Unfortunately, the session that is separated from us by only a curtain is sooooo loud, that the session with Duncan is almost unbearable :(

Categories: VMWare, VMWorld Tags:

My VMWorld schedule today

October 9th, 2012 No comments

20121009-122125.jpg

Categories: VMWare, VMWorld Tags:

VMWorld 2012 – performance, new features and best practices

October 9th, 2012 No comments

Just rushed into hall B2 , going to go through performance enhancements in vSphere 5.1. Ignore typos please – typing in an iPad and autocorrect is not the most uhhh tech friendly tool…

VSphere 5.1 targets
Big data
Low latency
Monster apps
Large scale deployments
View and vCloud director environments.

Big Data:
Monster VMs mean:
64vCpus (who does this with a Vm?)
1TB Ram (again – surely this would justify a physical?)
VMWare have managed to more than 1 million IOPs out of a single VM (cool and ridiculous)

Big new addition – exposure of new CPU counters in new techs like ivyBridge, SandyBridge and PileDriver

Low latency:
New dropdown available to label VMs as latency sensitive (VM behave accordingly) – use with caution . . And do NOT let the business know about this. Prioritises access to resource, but if overused, loses its effectiveness.

Platform recommendations:
Size VMs correctly
Use resource settings only if needed
Avoid affinity possible
Over provisioning is fine great!
Hyper threading is GREAT – use it.
Double check bios and power management settings

Reduced memory overhead:
VSphere 5.1 allows for swap file creation to reduce memory reservation for backed processes – saving about 1GB per host.
Can be configured from web client under system volumes – edit system swap settings.
Overcommit to about 20% as a guideline. Make sure to use ballooning, transparent page sharing, memory compression, host cache swapping and ESX or guest Level swapping.
When you start seeing the swapping use go up – reduce overcommitte.
Sizing VMs – use reservations as needed and try keep memory within NUMA domain.

Memory – consumed vs active:
Consumed – physical memory used by VM (good measure of actual usage at point in time)
Active is the amount recently touched.

Storage IO control enhanced in vSphere 5.1:
VSphere 5.1 can use percentage based thresholds instead of absolute latency values – this means better throughput on both slow storage as well as low latency for low latency storage,
SIOC monitors and controls the full storage stack latency

Storage DRS enhancements:
Interoperability with vCloud Director – including linked clone (with vCloud only)
Storage DRS correlation detector (so we won’t automatically move storage between data stores that are actually hosted on the same spindles – which would have no benefit)
Can be used with Auto-Tiering – but you would need to follow the storage vendor’s best practice.

Storage performance:
Now support 16Gb CPU – which has lower CPU cost / efficiency.

Adapters:
Jumbo Frames best case throughput improved by:
Hwscsi read 88% write 20%
Swscsi read 11% write 40%
NFS. Read 9% write 32%

Storage best practices:
Size accordingly and keep latency below 30ms
Snapshots are not free!
Use sioc and sdrs
Update storage firmware
Remember the old tricks (multipathing, block size, alignment, paravirtulaised scsi etc)

Networking virtualisation:
New features – VDS snapshots(snapshot your switch’s config), auto-Rollback of configs, port mirroring and net flow enhancements)

Use VDS – Network IO control (eg don’t let a vMotion kill the mic for everyone else)

New feature: SR-IOV – allow one nic to be presented as multiple separate logical adapters. This allows us to allow multiple VMs to directly use the physical NIC – reducing latency.

VXLAN – new feature
Deploy VMs where resources are available, then create a gigantic layer 2 network, making access ‘local’ – possibly a great tool for getting max use out of geographically dispersed vSphere networks that run business hours only – e.g. NY / London office

Networking best practice:
Be mindful of converged networks
Use distributed virtual switches

VMotion enhancements:
Shared nothing migration – no shared storage required and still able to migrate host and storage at the same time (cool)
Parallel storage vMotion – so we do a storage vMotion of a VM with 4 disks – possibly separated by affinity etc. this allows the copies of up to 4 vmdks at the SAME time (previously, copies were sequential) – there is only benefit when the vmdks are moving from different data stores, to different data stores.

vMotion best practices:
Use the latest version of vmfs. (5.x)
Keep vmknics on same subnet
Separate vmknics across multiple vmnics. VMotion will load balance the traffic

vCenter enhancements:
Web client WITH SSO
Wb client supports 300 concurrent connection
Can collect up to 80 million stats per hour – so max logging (level 4) for an environment of 1000 hosts, with 2000 data stores and 15000 VMs!

VCenter best practices:
Size correctly
Size the db correctly
Keep an eye logging levels, DB performance and networking connectivity between VC, DB hosts etc.
VM or physical is ok
32 hosts per cluster
Use resource pools and affinity rules in clusters as needed.

Categories: VMWare, VMWorld Tags:

VMWorld Barcleona 2012

September 19th, 2012 No comments

John Troyer (and Alex Maier) was kind enough to offer me a blogger’s pass (which as a contractor that has to take time off work to get to conferences etc is the only way I can afford to go) I have booked my flights and will be there for the Tuesday and Wednesday – couldn’t really afford 3 days unpaid – but here is my current schedule:

Subject Start Date Start Time End Date End Time
GS01  —  IT Transformation as the Enabler of Business Transformation and Delivering the Promise of the Software-defined Datacenter

09/10/2012

9:00 AM

09/10/2012

10:30 AM

GD41  —  vCloud Director Architecture, IntegraIon and Orchestration with Chris Knowles

09/10/2012

11:00 AM

09/10/2012

12:00 PM

INF-VSP1372  —  What’s New with vSphere

09/10/2012

12:30 PM

09/10/2012

1:30 PM

INF-VSP1329  —  PowerCLI Best Practices:  The Return!

09/10/2012

2:00 PM

09/10/2012

3:00 PM

INF-BCO1159  —  Architecting and Operating a VMware vSphere Metro Storage Cluster

09/10/2012

3:30 PM

09/10/2012

4:30 PM

GS02  —  Empowering the Workforce of Tomorrow, That’s Here Today

10/10/2012

9:00 AM

10/10/2012

10:30 AM

INF-VSP1353  —  vCenter: A Technical Deep Dive

10/10/2012

11:00 AM

10/10/2012

12:00 PM

VCAP Datacenter  —  VMware Quickstart: VCAP Datacenter

10/10/2012

12:30 PM

10/10/2012

1:30 PM

OPS-CIM2179  —  Transforming Your Cloud with VMware:  Day One – Building Your Cloud

10/10/2012

2:00 PM

10/10/2012

3:00 PM

INF-STO1545  —  Architecting Storage DRS Datastore Clusters

10/10/2012

3:30 PM

10/10/2012

4:30 PM

INF-VSP1252  —  What’s New with vSphere 5.1 – ESXCLI & PowerCLI

10/10/2012

5:00 PM

10/10/2012

6:00 PM

I am still toying with the extra day off as there are some sessions on Friday that I would like to attend and of course, I don’t really want to miss the party! If you haven’t booked – best get booking!!  

Categories: VMWare, VMWorld Tags: