VMware虚拟化最佳实践及规划.ppt

虚拟化最佳实践及规划,议程,应用实施范围考虑服务器采购考虑虚拟机部署考虑管理维护考虑,议程,应用实施范围考虑服务器采购考虑虚拟机部署考虑管理维护考虑,应用实施范围总体原则,不适合采用虚拟化的应用具有特殊硬件访问要求的应用高性能图形显卡-不适用虚拟化 特殊的串/并行加密设备-不适用虚拟化USB设备连接需求-可能不适用，可采用外置USB设备代替，需经过测试即使在高配置的服务器上仍然具有很高负载的应用-可能不适用，需分析当前服务器配置情况可以采用虚拟化的应用除上述不适合采用虚拟化的应用之外的所有应用可根据应用迁移的复杂程度决定虚拟化先后顺序较易实现P2V的应用可先做迁移，如可用Converter工具直接迁移的应用较难或不能做P2V迁移的应用可考虑采用重新安装方式后迁根据管理的需要决定是否做虚拟化虚拟化转变过程对现有业务的影响程度转变为虚拟化后对现有管理的影响程度部门之间协调的难易程度,虚拟化宿主服务器的部署类型,垂直扩展与水平扩展部署模式不同资源池的“量子化”模型,不同资源池的类型,垂直扩展的主机模式提供更大的连续性资源空间更容易满足不同负载的吻合性要求可提供更高的资源利用率水平扩展的集群主机模式更像是一组小池子的集合多组小容量资源池需要更多的监控管理模块化使用既有优点也有缺点,影响虚拟化部署的参数体系,功能的多样化,服务器的重要性,独立服务器,不重要,重要,(独立的,本地存储等等),(群集的,多主机的等等),服务器农场,(水平扩展服务器群集，公用服务器等等),(后端办公,本地应用等等),负载约束,技术约束,商业约束,困难度增加,负载约束,不同的资源组都需要分别考虑CPU利用率磁盘 I/O网络 I/O内存利用率虚拟化造成的额外负载通常也要做一定考虑，如磁盘和网络的I/O会增加CPU的负担iSCSI存储访问也会增加CPU负担运维的周期性负载变化也必须考虑进来月末负载变化年末负载变化,技术约束,技术约束通常主要是指：兼容性（指系统/应用的兼容性）关联性（如系统是摩格逻辑组的一部分）大部分环境下，这些约束包括了：网络连接（子网段级别）应用之间的互连性相关的存储使用技术所用的硬件和外设软件支持度和认证这些约束条件根据虚拟化在内核上下实现的不同而有所不同共享与分离OS镜像模式,商业和流程的约束,规模较小或集中的情况下容易被忽略的约束在实验室的测试环境可以不考虑，但生产环境必须要考虑在虚拟化中常见的商业和流程约束包括：维护窗口和冻结改变地理位置和其他物理限制运维环境，安全区域，应用分层部署考虑商业组织，部门以及客户法规政策的考虑与限制忽视这些约束条件将可能导致不可预知的结果具体情况具体分析，根据目标制定计划,议程,应用实施范围考虑服务器采购考虑虚拟机部署考虑管理维护考虑,虚拟化中使用的硬件应满足兼容性列表要求,所有用于实现VMware VI3虚拟架构解决方案的设备，包括：服务器系统、存储系统、IO卡设备等，应满足VMware VI3产品兼容列表的要求，最新的兼容列表可从如下的连接找到：服务器系统兼容列表 HCL:Systems Compatibility Guide For ESX Server 3.5 and ESX Server 3i存储系统兼容列表HCL:Storage/SAN Compatibility Guide For ESX Server 3.5 and ESX Server 3iIO卡设备兼容列表，包括网卡、FC HBA卡和iSCSI HBA卡等 HCL:I/O Compatibility Guide For ESX Server 3.5 and ESX Server 3i,ESX服务器硬件配置考虑要点 CPUs,ESX 调度CPU周期满足虚拟机和Service Console的处理请求可用的CPU目标数量越多，ESX管理这个调度机制的效果越好（单台服务器配置8个以上的CPU核会有最好的效果）超线程技术并不能提供等同于多核处理器的好处；建议关闭CPU的超线程功能（如果有的话）使用具有EM64T能力的Intel VT 或AMD V 技术的CPU可以同时支持运行32位和64位的虚拟机采用同一厂商、同一产品家族和同一代处理器的服务器组成的集群，可以获得最好的VMotion兼容能力ES的Enhanced VMotion兼容性扩大了原有VMotion的兼容能力-Alleviating Constraints with Resource Pools Live Migration with Enhanced VMotion,-参见Best Practices for Successful VI Design,ESX服务器硬件配置考虑要点-内存,内存资源往往比CPU资源更会成为潜在的瓶颈在某些时候，虚机环境的内存使用量可能会超过物理内存值：Host swap file(尽量少用以获得最佳性能)Transparent Page Sharing（多个虚机共享相同内存页面）注意服务器特定的内存配置要求DIMM sizes,bank pairing,parity,upgrade considerations(mix and match or forklift replacement)尽可能将服务器配置到最大内存，采用最大容量的内存条（特别是当没有配满全部内存条时）,-参见Best Practices for Successful VI Design,虚拟架构的基本网络连接部件组成,(Management virtual machine),(Vmotion,iSCSI,NFS),(VM connectivity),Port Group,Port Group,Port Group,ESX服务器硬件配置考虑要点-网络,-参见Best Practices for Successful VI Design,最少配置一个虚拟交换机，测试环境可用2个虚拟交换机，生产环境建议最少配置3个虚拟交换机虚拟交换机可同时支持3种类型的端口组(Service Console,VMkernel,VM)建议将Service Console、VMkernel和虚机端口组各自使用自己的虚拟交换机可用VLAN技术分割不同的端口组对于使用VMotion和DRS功能的服务器集群，网络配置应该相匹配（虚拟交换机的数量与网络卷标名应保持一致）ESX服务器Service Console使用固定IP，配置正确的speed和duplex。,ESX服务器硬件配置考虑要点-网络-虚拟交换机和端口组,-参见Best Practices for Successful VI Design,ESX服务器，虚拟交换机，物理网卡出于冗余的考虑，每个虚拟交换机建议至少分配两个物理网卡每个ESX服务器的物理网卡/口数量取决于准备配置的虚拟交换机的数量如果3种类型的端口组（SC,VMkernel,VM）都在不同的虚拟交换机上，生产环境建议至少6个物理网卡/口如果给包含虚拟机端口组的虚拟交换机分配更多的物理网卡/口，可以获得负载均衡的好处,ESX服务器硬件配置考虑要点-网络基本组件,-参见Best Practices for Successful VI Design,物理网卡/口与物理交换机同一个虚拟交换机上的不同物理网卡/口应连接到不同的物理交换机上将一个集群中所有服务器的VMotion功能端口组所使用的物理网卡/口都连到同一套物理交换机上（同样遵循上述第一条规则）,ESX服务器硬件配置考虑要点-与物理网络的连接,-参见Best Practices for Successful VI Design,Example 1:Blade Server with 2 NIC Ports,vSwitch,vmnic0,SC,vmkernel,Active,Standby,vmnic1,Candidate Design:Team both NIC portsCreate one virtual switchCreate three port groups:Use Active/Standby policy for each portgroupPortgroup1:Service Console(SC)Portgroup2:VMotionPortgroup3:VM trafficUse VLAN trunkingTrunk VLANs 10,20,30 on each uplink,Portgroup1VLAN 10,Portgroup3VLAN 30,Portgroup2VLAN 20,VLAN Trunks(VLANs 10,20,30),Example 2:Server with 4 NIC Ports,vmnic0,SC,vmkernel,Active,Standby,vmnic1,Candidate Design:Create two virtual switchesTeam two NICs to each vSwitchvSwitch0(use active/standby for each portgroup):Portgroup1:Service Console(SC)Portgroup2:VMotionvSwitch1(use Originating Virtual PortID)Portgroup3:VM traffic#1Portgroup4:VM traffic#2Use VLAN trunkingvmnic1 and vmnic3:Trunk VLANs 10,20vmnic0 and vmnic2:Trunk VLANs 30,40,Portgroup4VLAN 40,VLANs 10,20,vSwitch0,Portgroup1VLAN 10,Portgroup2VLAN 20,vSwitch1,vmnic2,vmnic3,Portgroup3VLAN 30,VLANs 30,40,Example 3:Server with 4 NIC Ports(Slight Variation),vmnic0,SC,vmkernel,Active,Standby,vmnic1,Candidate Design:Create one virtual switchCreate two NIC teamsvSwitch0(use active/standby for portgroups 1&2):Portgroup1:Service Console(SC)Portgroup2:VmotionUse Originating Virtual PortID for Portgroups 3&4Portgroup3:VM traffic#1Portgroup4:VM traffic#2Use VLAN trunkingvmnic1 and vmnic3:Trunk VLANs 10,20vmnic0 and vmnic2:Trunk VLANs 30,40,VLANs 10,20,vSwitch0,Portgroup1VLAN 10,Portgroup2VLAN 20,vmnic2,vmnic3,Portgroup3VLAN 30,VLANs 30,40,Portgroup4VLAN 40,Servers with More NIC Ports,More than 4 NIC PortsDesign ConsiderationsWith Trunks(VLAN tagging):Use previous approach and scale up to meet additional bandwidth and redundancy requirementsAdd NICs to NIC team supporting VM traffic VLAN Tagging always recommended,but options if NICs available:Dedicated NIC for VMotionAt least one NICDedicated NICs for IP Storage(NFS and/or iSCSI)Usually two teamed NICs(consider IP-hash&etherchannel if multiple destinations and Multi-Chassis Etherchannel employed on physical switches)Dedicated NIC(s)for Service ConsoleAt least two for availabilityNote:easy to consume many physical NICs and switch ports if not using VLAN tagging,ESX服务器硬件配置考虑要点-存储,应尽可能采用外置共享磁盘阵列存放虚拟机文件ESX服务器内置硬盘应有充分的冗余，建议采用RAID1ESX服务器自身对硬盘要求，安装时的Partition划分：不建议用安装时的自动硬盘划分方法，因为/、/var、/home会放再同一个目录下，当/(root)满了时，ESX服务器会发生严重问题。建议：/boot50 到100 MB(Primary Partition)/8.0 到18GB(Primary Partition)(swap)2倍的Service Console内存,建议固定使用1.6G/var4GB 或更大建议足够的ESX服务器程序空间大小为18GB本地端的ISO以及其他文本文件的存放空间要考虑,存储对于虚拟机的呈现方式,7,VM 层,数据存储,存储阵列,SCSI 控制器,虚拟磁盘呈现为 SCSI 控制器 SCSI 控制器显示为 BUS 或 LSI Logic 磁盘控制器 一个 VM 可具有 1 到 4 个虚拟 LSI Logic 或 BusLogic SCSI 适配器 每个 SCSI 适配器包含1 到 15 个虚拟 SCSI 存储设备 虚拟磁盘驻留在可格式化为 VMFS、NFS 或裸磁盘的数据存储中 文件系统类型由底层物理磁盘驱动 器确定,VMFS,NFS,FC,iSCSI,NAS,卷、数据存储和 LUN,卷,存储阵列,数据存储,8,LUN 1020 GB,LUN 是一个逻辑空间可由存储阵列的整个空间创建，也可由其中的 部分空间创建 LUN 映射到 ESX 后即成为卷 当卷被格式化为某种文件系统之后即成为数据存储 不能在同一个 LUN 中混用不同类型的文件系统 每个 LUN 对应一个 VMFS 卷,虚拟机内容位于数据存储中,数据存储,ESX 数据存储类型：VMware 文件系统（VMFS）使用 VMFS 的 RDM 网络文件系统（NFS）,卷,VM 内容,ESX 主机,数据存储采用某种文件系统格式 可以像操作文件一样操作数据存储 每个系统具有 256 个 VMFS 数据存储 每个系统具有 8 个 NFS 数据存储 ISO 映像、VM 模板和软盘映像,9,虚拟机内容,10,文件夹/子目录,数据存储类型,数据存储,VMFS,数据存储,VMFS,数据存储,NFS,IP 网络,VM 内容,VM 内容,FC 交换机,光纤通道SAN 磁盘阵列,iSCSI SAN磁盘阵列,NAS磁盘阵列,VM 内容,VM 3,VM 3,12,本地 SCSI,VMDK,IP 交换机,ESX服务器建议配置-新购,为了尽可能的发挥虚拟化的作用，最大限度的利用单台服务器的资源，建议用于虚拟化宿主服务器的配置应达到或超过如下标准：,从性价比和可用性考虑，不建议在单路服务器上部署虚拟化,虚拟化宿主服务器建议配置-现有,对于目前业内用的比较多的四路服务器，建议如下：,四路单核服务器：运算能力较弱，虚机数量应控制在10个以内，内存配置建议在12GB-16GB；四路双核服务器：运算能力中等，虚机数量可做到10-15个左右，内存配置建议在16GB-24GB；四路四核服务器：运算能力强劲，虚机数量可做到15-30个左右，内存配置建议在24GB-32GB。,VC服务器最佳配置建议,处理器：2.0GHz或更高的Intel或AMD x86处理器，VC支持多处理，可支持至多2个CPU。内存：最低需求为2GB，假使数据库和VC安装于同一台，建议增加至4GB。磁盘空间：最小为560MB，建议2GB。网卡：建议用Gigabit。最低硬件配置-单个2GHzCPU，2GB内存，千兆网口可支持20个同时连接，管理50台物理机，1000个虚拟机左右建议配置-双CPU，4GB内存，千兆网口可支持50个同时连接，管理200台物理机，2000个虚拟机左右,议程,应用实施范围考虑服务器采购考虑虚拟机部署考虑管理维护考虑,虚机个数的规划,单台服务器所能支持虚机数量的决定因素：服务器的硬件配置CPU性能-多核高主频技术使得CPU成为性能瓶颈的可能性越来越低内存大小-做为硬指标的内存，配置越高，所能支持的虚机数量越多网络端口-千兆网环境已很普遍，网络带宽大多有保证，更多从管理角度来考虑HBA卡-磁盘访问性能对虚机数量有一定影响，建议采用4Gb或8GbHBA卡以减少链路影响本地磁盘-内置磁盘的可用性及IO吞吐能力均较弱，不建议在其上存放虚拟机，推荐使用外置高性能磁盘阵列应用负载大小由于物理服务器资源自身的最大限制，应用负载越大，所能同时运行的虚机数量越少建议将不同应用访问特性的应用混合部署在同一物理服务器上灵活运用DRS和VMotion技术可将物理机与虚机的比率关系调到最优考虑到HA及DRS所要求的资源冗余，所有运行虚机在正常负载下，总体资源使用率不超过三分之二会比较合适经验值：双路四核10个虚机左右，四路四核15-30个虚机（仅为参考）,虚机资源的分配-CPU、内存,CPU分配原则：尽量使用最少的vCPUs，如果是单线程应用，不支持多线程处理，请不要使用virtual SMP虚拟CPU数量不要等于或超过物理CPU核数，如双路双核服务器配置的虚机最多使用两个虚拟CPU当配置虚拟机的时候须了解ESX服务器本身也有一些overhead。需注意不要超过所有虚拟机使用率和所有vCPU汇总数目。观察”idle loop spin”功能参数，某些操作系统当它们闲置时，并不会真正的释放virtual CPU。确认配置了单一处理器的虚拟机为”UP HAL/kernel”，多处理器的虚拟机必须设定为”SMP HAL/kernel”。内存分配原则：内存总量为在资源评估后，计算虚拟机评估结果所需实际物理内存的总和，其他由于应用程序而产生的更多内存需要可以用ESX的磁盘内存来解决关键应用可考虑固定内存的方法以保证性能的稳定性,DRS Best Practices:Hardware Configuration,Ensure hosts are CPU compatibleIntel vs AMDSimilar CPU family/SSE3 statusEnhanced VMotion Compatibility(EVC)“VMware VMotion and CPU Compatibility”whitepaperCPU incompatibility=limited DRS VM migration optionsLarger Host CPU and memory size preferred for VM placement(if all equal)Differences in cache or memory architecture=inconsistency in performance,DRS Best Practices:Cluster Configuration,Higher number of hosts=more DRS balancing optionsRecommend up to 32 hosts/clusterMay vary with VC server configuration and VM/host ratioNetwork configuration on all hostsVMotion network:Security policies,VMotion nic enabled,GigE network,etcVirtual Machine network present on all hostsVM datastore shared across all hostsVM floppy/CD connected to host device,DRS Best Practices:VM Resource Settings,Reservations,Limits,and SharesShares take effect during resource contentionLow limits can lead to wasted resourcesHigh VM reservations may limit DRS balancingOverhead memoryUse resource pools(RP)for better manageabilityVirtual CPUs and Memory size High memory size and virtual CPUs=fewer migration opportunitiesConfigure VMs based on need,DRS Best Practices:Algorithm Settings,Aggressiveness thresholdModerate threshold(default)works well for most casesAggressive thresholds recommended ifHomogenous clusters andVM demand relatively constant andFew affinity/anti-affinity rulesUse affinity/anti-affinity rules only when needAffinity rules:closely interacting VMsAnti-affinity rules:I/O intensive workloads,availabilityAutomatic DRS mode recommended(cluster-wide)Manual/Partially automatic mode for location-critical VMs(per VM)Per VM setting overrides cluster-wide setting,HA Best Practices-Setup&Networking,Proper DNS&Network settings are needed for initial configurationAfter configuration DNS resolutions are cached to/etc/FT_HOSTS(minimizing the dependency on DNS server availability during an actual failover)DNS on each host is preferred(manual editing of/etc/hosts is error prone)Redundancy to ESX Service Console networking is essential(several options)Choose the option that minimizes single points of failureGateways/isolation addresses should respond via ICMP(ping)Enable PortFast(or equivalent)on network switches to avoid spanning tree related isolationsNetwork maintenance activities should take into account dependencies on the ESX Service Console network(s)VMware HA can be temporarily disabled through the Cluster-Edit Settings dialogValid VM network label names required for proper failoverVirtual machines use them to re-establish network connectivity upon restart,HA Network Configuration,A single service console network with underlying redundancy is usually sufficient:Use a team of 2 NICs connected to different physical switches to avoid a single point of failureConfigure vNics in vSwitch for Active/Standby configuration(rolling failover=“yes”,default load balancing=route based on originating port ID)Consider extending timeout values&adding multiple isolation addresses(*see appendix)Timeouts of 30-60 seconds will slightly extend recovery times,but will also allow for intermittent network outages,Network redundancy between the ESX service consoles is essential for reliable detection of host failures&isolation conditions,HA Network Configuration(Continued),HA will detect and use a secondary service console networkAdding a secondary service console portgroup to an existing VMotion vSwitch avoids having to dedicate an additional subnet&NIC for this purposeAlso need to specify an additional isolation address for the cluster to account for the added redundancy(*see appendix)Continue using the primary service console network&IP address for management purposesBe careful with network maintenance that affects the primary service console network and the secondary/VMotion network,Beyond NIC teaming,a secondary service console network can be configured to provide redundant heartbeating&isolation detection,HA Best Practices Resource Management,Larger groups of homogenous servers will allow higher levels of utilization across an HA/DRS enabled cluster(on average)More nodes per cluster(current maximum is 16)can tolerate multiple host failures while still guaranteeing failover capacitiesAdmission control heuristics are conservatively weighted(so that large servers with many VMs can failover to small servers)To define the sizing estimates used for admission control,set reasonable reservations as the minimum resources neededAdmission control will exceed failover capacities when reservations are not set;otherwise HA will use largest reservation specified as the“slot”size.At a minimum,set reservations for a few virtual machines considered“average”Admission control may be too conservative when host and VM sizes vary widelyPerform your own capacity planning by choosing“Allow virtual machines to be powered on even if they violate availability constraints”.HA will still try to restart as many virtual machines as it can.,议程,应用实施范围考虑服务器采购考虑虚拟机部署考虑管理维护考虑,Impact of VirtualCenter Downtime,-参见Bulletproof VirtualCenter-A Guide to Protecting VirtualCenter,VirtualCenter Components,VirtualCenter Server,Web Access,License Server,AD Domain Controller,DNS Server,Database Server,-参见Bulletproof VirtualCenter-A Guide to Protecting VirtualCenter,VirtualCenter Recommended Collocation,Collocation of VirtualCenter components is desirable for most environmentsFocus of this session is on providing protection for these componentsIndustry standard solutions assumed for other components,One Server,Physical or Virtual,VirtualCenter Server,Web Access,License Server,AD Domain Controller,DNS Server,Database Server,-参见Bulletproof VirtualCenter-A Guide to Protecting VirtualCenter,VirtualCenter Components(Additional Details),VirtualCenter Service:almost statelessInformation about inventory stored in the databaseSome state files stored locally on VirtualCenter serverWeb AccessNo state informationLicense ServerLicense file stored locally14 day Grace period if unavailable,-参见Bulletproof VirtualCenter-A Guide to Protecting VirtualCenter,VirtualCenter Local Configuration Files,One Server,Physical or Virtual,VirtualCenter Server,Web Access,License Server,Database Server,SSLCertificate,License File,Config.File,UpgradeFiles,-参见Bulletproof VirtualCenter-A Guide to Protecting VirtualCenter,Step 1 for High Availability:Protect the Database,Database outage will terminate VirtualCenter serviceAs of VirtualCenter 2.0.1 Patch 2,Windows Service Manager will automatically attempt to restart it every 5 minutes,indefinitelyVirtualCenter Database should be independently installed and managedFor local availability use the preferred mechanism for the type of database being used(VMware HA,MSCS,Database specific mechanisms)For disaster recovery,database should be replicated to a remote site as part of an overall DR plan,-参见Bulletproof VirtualCenter-A Guide to Protecting VirtualCenter,VC,VC,Bang!,Failover,Step 2 for High Availability:Protect VirtualCenter,VMware HA and Microsoft Cluster Services(MSCS)are the two most popular optionsOther 3rd party solutions possible*Supported directly by 3rd party,Option a):VMware HAVirtual instances onlySubject to shared storag