存储厂家EMC云计算存储解决方案 AtmosTechnicalOverview.ppt

《存储厂家EMC云计算存储解决方案 AtmosTechnicalOverview.ppt》由会员分享，可在线阅读，更多相关《存储厂家EMC云计算存储解决方案 AtmosTechnicalOverview.ppt（54页珍藏版）》请在三一办公上搜索。

1、EMC Atmos:Technical Overview,EMC,Agenda,Atmos OverviewAtmos OfferingsAtmos hardwareArchitectural Overview of AtmosOperational EfficiencyAccess MethodsRole of policiesAtmos Online,Where does Cloud fit?,Market leading storage solutions for all information and applications,Atmos Solution Offerings,Bund

2、led hardware and software solutionLarger footprints(up to 720 TB in one frame)Ease of configurationSoftware-only solutionSmall footprintsVMWare installSupports existing hardwareDeployment terminologyAtmos cloud:Comprised of one to many locationsEach location is a“Resource Management Group”(RMG)Each

3、RMG is comprised of one or more“Installation Segments”(IS)An IS equates to the private network connecting several Atmos“nodes”Rack Two or more Atmos VMs on the same private subnet,Atmos Next Generation Hardware,Dense Compute1:15 Server-to-drive ratioNow up to 480 TB total capacityLower Power/BTU at

4、2x Capacity,Small/Medium Scale Compute1:15 servers-to-disks60 TB Cloud Entry PointNow up to 240 TB total capacityLower Power/BTU at 2x Capacity,Capacity1:60 Server-to-drive ratioNow up to 720 TB total capacityLower Power/BTU at 2x Capacity,New,Intel Xeon 5500“Nehalem”Microarchitecture,2 TB Low Power

5、 SATA Drives,New,Hardware Design Philosophy,Industry standard componentsStandard data center environmentsFactory configured,racked and cabledFlexible,cost effective service modelOptional HA and redundancyUser serviceable30 drives per 3U drawer Up to 720 TB per rackWS2-120,WS2-240 or WS2-360 configur

6、ations,WS2-360:Dense Scale Capacity,Up to 720TB total capacity1 or 2 TB Drives44 U“Hulk”cabinet1:60 server to drive ratio6 servers/24 bullet DAEsPartial Rack will be supported Q24-6 Nodes,2 Node increment1TB Drive:240,360 TB2TB Drive:480,720 TBGbe connectivity1 Allied Telesis Gbe switchRedundant pow

7、er and coolingEnvironmentals(1 TB)82”height 24”width43”depth2548 lbs weight 10,290 W power35,120 BTU/hr,WS2-240:Dense Scale Compute,Up to 480TB total capacity1 or 2 TB Drives44 U“Hulk”cabinet1:15 server to drive ratio16 servers/16 bullet DAEsPartial Rack will be supported Q24 16 Nodes,2 Node increme

8、nt1TB Drive:60,90,120,180,210,240 TB2TB Drive:120,180,240,300,360,420,480 TBGbe connectivity2 Allied Telesis Gbe switchesRedundant power and cooling Environmentals(1 TB)82”height24”width43”depth2282 lbs weight10,020 W power34,198 BTU/hr,WS2-360/WS2-240 Serviceability and Accessibility,Patented expan

9、dable drawer provides full access to embedded components,Full rear access,No special cooling requirements,Clean,rear-only cabling and automated cable management systems,WS2-120:Small Scale Compute,Up to 240 TB total capacity1 or 2 TB drives40 U“Titan”cabinet1:15 server to drive ratioPartial Rack in

10、Q1:4,6,or 8 Nodes1TB:60TB-4 Node,90TB-6 Node,120TB-8 Node2TB:120TB 4 Node,180 TB 6 Node,240 TB 8 NodeGbE connectivity1 Allied Telesis IP x9924 switch2 PDUsEnvironmentals(1 TB)70”height 24”width40”depth1,356 lbs weight4,880 W power16,655 BTU/hr,Additional Cabinet Power Information,SOAP/REST,DNS Load

11、Balancer,Reference Architecture,Access Points,CIFS/NFS/IFS,Location 3“Tokyo”,Location 2“London”,Location 1“Boston”,Access Points,TCP/IP,TCP/IP,PolicyManagement,JobService,StorageService,Atmos High-Level Architecture,Atmos Native Client Access API,MetadataService,Applications,REST,SOAP,Apache HTTPD,I

12、nstallable File System(IFS),SystemManagement,MetadataLocationService,ResourceManagementService,Administration,1-Client access,2-Data,5-Management,4-Discovery,3-Metadata,Atmos SoftwareCollection of distributed servicesClient access(1)Data(2)Metadata(3)Discovery(4)Management(5)RPC-XML like Protocol,x8

13、6 servers,Dense storage,VMware ESX,Atmos Virtual Edition/Atmos Hardware,Clients/Management,NFS,CIFS,Reference Architecture Creating Objects,“London”,Application Server Farm,DNS Load Balancer,Application makes API call to create object,Load balances to“access node”,Meta data written“locally”.,and syn

14、chronized,Policy manager makes 2 synchronous replicas,“Boston”,Object identifier returned to application,Reference Architecture Reading Objects,“London”,Application Server Farm,DNS Load Balancer,Application makes read object call,Load balances to“access node”,Meta data server queried,Optimal object

15、found,Data read from copy,“Boston”,Object data and meta data returned to application,Atmos Tenancy Model,Foundation for cloud storage architecture Conceptual grouping of resources Hierarchical model enables flexible application integrationPlatform to support millions of users Provides robust reporti

16、ng and resource managementImproves operational efficiency Manage access and resources per tenant/subtenantAllows optimal policy definition and management Helps manage node services,Secure Multi-Tenancy,Access control,auditing,separationMultiple“customers”can share the same cloudIsolation between set

17、s of customers(at the Tenant and Sub-tenant level)Maui tenant capabilitiesMultiple Tenants,each with a unique Tenant IDUnique policies per TenantLogical separation of all resources availableNote:servers are dedicated to one TenantExamples of multiple Tenants sharing a single Atmos cloudCustomers(cli

18、ents of an organization)Business units(within the same organization)Applications(CIFS backups vs.REST access)Geographic regions(e.g.Australia,Germany,United States)Tenants can be divided into Sub-TenantsBy default,each Tenant has a Sub-Tenant with the same Tenant ID,Atmos Tenancy Model,Conceptual gr

19、ouping of resources Tenants,Sub-tenants and Users Physical and network separation of Tenants,logical separation of Sub-tenants(each have a unique system-wide ID)Unique policies per TenantSeparate admin management,user management and Tenant-defined policies per Sub-TenantSupports thousands of custome

20、rs,departments,applications or users in one cloud,Supports internal/external service providersSupports tenancy across many locations while providing a unified name space within the TenantSupports optimal policy definition for different groups or applications,providing operational efficiencyAllows gr

21、anular management of access and resources per Tenant,Subtenant and UserProvides robust reporting,Boston RMG,NODE 1,NODE 2,NODE 3,NODE 4,NODE 5,NODE 6,NODE 7,NODE 8,London RMG,NODE 1,NODE 2,NODE 3,NODE 4,NODE 5,NODE 6,NODE 7,NODE 8,Tenant A,Tenant B,CIFS Sub-tenant,User n,User 1,CIFS Users,NFS Users,

22、NFS Sub-tenant,User n,User 1,Sub-tenant2,User n,User 1,WS Users,WS Users,Sub-tenant1,User n,User 1,Atmos Policy,Key feature of Atmos Cloud storageMaps business requirements Defined and managed per TenantSub-tenants inherit policies from TenantsOne default policy per tenantOptimal and efficient stora

23、ge managementMaps business objectives to replication and location requirementsIncrease operational efficiencySet it and forgetValue added services are defined per policyData-path serviceData at rest serviceLocation attributesSS attributesExpiration and retentionInitial Design and Implementation of p

24、olicies is vital,Policy-based Information ManagementIncreasing Operational Efficiency,Audio Transcripts,Digital Images,Object metadata matches policies Policies map to business objectives,Policies drive actions Where to store information What content services to apply The number of copies,Gold:2 syn

25、chronous copies Boston1 asynchronous copy Seattle1 asynchronous copy MiamiBronze:1 synchronous copy MiamiDe-dupe,compression1 asynchronous copy Seattle,Availability=Gold,Availability=Bronze,“.mov”,“image”,etc.,“.mp3”,“audio”,etc.,Custom Metadata,Replicated by policyacross your Enterprise,Retained an

26、d archived to the cloud,Policy-Driven Management,Additional Data Services,CompressionReduction of physical space required for object storageDe-duplicationReduction of replicated identical objectsSpin-down“Green”,power efficient method for long term storageFAST/StripingData striping within nodes or a

27、cross nodes for higher throughputAuto-configuration and auto-healingInstall once just add capacity,Node-levelData Services,Storage Server Attributes,Storage Server Attributes are set at install timeDefined per nodeThey cannot be changed on the flyAttributes must be defined on two levelsData Placemen

28、t Action Type(Transformations)Pick one of each for node profile,Data Placement,Action Type,Optimal,Green,Fast,Balanced,None,Compression,Deduplication,Checksum,Compress+Dedupe,Fine tune your environment,GeoProtect Dial the right performance and protection levels to your contentTwo implementation meth

29、ods to choose from,Implement both to maximize content distribution efficiency and cost,Managing storage at Global Scale A New Approach,1 Copy,2 Copies,N Copies,N,By Maintaining:,Susceptible to failures of the system,site,and network connectivityMay not meet required performance because limits of dev

30、ice and or location of access,Improves resilience to failures of the system,site,and network connectivity Potentially improves performance of remote access,Greatly improves resilience to failures of the system,site,and network connectivity Greatly improves performance of remote access,Disperse Copie

31、s Address Geographic Performance and Reliability,Managing storage at Global Scale A New Approach,Erasure Codes:Location specific content hardening Increase content durability with less overheadHigher Fault Tolerance(tolerates k faults)with less storage cost(n/m)Availability measures if data stored o

32、n a system can be returned promptly(i.e.no special action required to generate the data)Durability measures if data stored in a system is not lost due to failures(i.e.data is available but special action may be needed),Erasure Codes:At a Glance,Object written into m data segmentsand k coded segments

33、Determine where to store segmentsObject can still be retrieved when K components are unavailable or failCannot recreate the object without all required segments,Atmos Erasure Code Implementation,Objects are divided into m data fragments and k coding fragmentsSupports two configurations9/12 configura

34、tion=m=9,k=3(Default)Tolerates up-to three drive failures33%storage overheadPerformance overhead is under review10/16 configuration High fault tolerance(up-to 6 drive failure)60%storage overheadPerformance overhead is under reviewFragments are stored in independent fault domains1.3 release has the a

35、bility to choose fault domainsBased on replication location configuration Fragments can be distributed across RMG Fragments can be distributed across the System,Erasure Code Encode(9/12 configuration),Encode,Write,9 fragments,Encoded fragments,M=9,K=3,Objects are divided into m=9 data fragments and

36、k=3 coding fragmentsK=3 coding fragments are computed using Cauchy Reed-Solomon codeFragments are stored in independent fault domainsEach fragment is stored on a different diskRMG or system level fault domain optionsOverhead=k/(m)Ex:=3/(9)=33%Encode operations has overhead on CPU Fragment level cons

37、istency and integrity checks,Configuring Erasure Codes,Click“Erasure Code”Two additional fields appearAlgorithm:CRS(Cauchy Reed-Solomon)Fragmentation:9/12 or 10/16,Policy Layout Replica Options,Standard ReplicasSingle instance of the data perSingle drive Erasure Coded ReplicasPolicy based striped Re

38、plicasSpecify a number of Storage Servers to stripe acrossSpecify a striped depthFederated ReplicasWith an Atmos Online account configured a“federated”location is allowed,Object Recovery:Drive Failure,BenefitsProvides proactive object recovery in the event of a disk failureSupports multiple drive fa

39、iluresDoes not require drive replacementRecovery distributes objects within the same RMGAlerts and GUI provide information Recovery start timesObjects impactedCompletion status,Disk Failure Alert,Recovery Process has Begun,Alert for Impacted Objects,Recovery Progress,Flexible Access Methods,Web serv

40、ices accessSOAP:Simple Object Access ProtocolREST:REpresentational State TransferFile services accessNFS:Network File SystemCIFS:Common Internet File SystemIFS:Installable File System,CIFSNFS,HTTP,Cross Platform(via SOAP/REST),Linux/Unix(via NFS),Windows(via CIFS),Linux w/Fuse RPM(via IFS),IFS,WSSOA

41、PREST,AtmosNativeClientAPI,Overview,Installable(Atmos)File SystemNative Atmos file systemSame global file system code and library as on Atmos nodesAllows an external client act as the Atmos Client Services that runs on an Atmos node.IP-based communicationRequires FUSE and RHEL5,AtmosFS RPM,AtmosFS,m

42、auifs,IFSRHEL5 support only32 or 64 bitRequires fuse 2.7.4Speaks native AtmosAtmosFS servers are part of the cloudAtmos native file systemMounted to a subtenantClient mountpoint:/mnt/mauifsEven though it is/mnt/mauifs/TID/SIDMultiple AtmosFS clients may point to the same subtenantIFS is inherently c

43、lusteredImproves performance and parallelismAtmos ports must be openCan act as a gateway for NFS/CIFSMatch IFS version to Atmos version,RHEL5 Server,“Boston”,Application,Q1:Pushing a File to Atmos:FS Client,MDLS,MDS,RM,SS,SS,SS,ext3,ext3,ext3,MDS,PM,RM,Linux OS,/mnt/mauifsMaui client,1,1,1.File pass

44、ed to Maui mount point,2,2.Client queries RM for MDLS,4,4.Create request issued to MDS,5,5.Policy queried for layout,6,LSO,6.Layout(LSO)and OID returned to Client,7,7.Client writes data to selected resources Proprietary protocols,3,3.Client queries MDLS for MDS that owns the parent directory,NFS and

45、 CIFS Access,NFS or CIFS configured per Atmos NodeNFS service or SMB service startedLayered on top of the AtmosFS(IFS)that runs on each nodeNo automatic failover of access nodes,Unix Clients,Windows Clients,CIFS Configuration,Atmos exports CIFS at node(tenant)level Tenant or subtenant admin manages

46、the shares Regardless of the access method,all objects are defined using global unique identifier(object ID)All files are defined using a global,unique identifierTranslation from traditional namespace to the object ID transparentSub-tenant sharesAbility to access CIFS share if multi-subtenant access

47、 is enabledLeverage same authentication model that of TenantsCIFS User Management is managed using AD integrationAtmos CIFS share ACL detailsDirectory and file level permission inheritanceACL inheritanceDefault permissions configurationEase of CIFS share management and configurationMulti-level admin

48、 control based on sub-tenant and or tenant access rights,CIFS Access Architecture,EMC Atmos 1.2.x,Share Y,Tenant ID(Tenant 2),Share A,Share B,CIFS Clients,Access Point,Access Point,Access Point,Access Point,Access Point,Access Point,Atmos Linux Driver,SMBd,FUSE 2.4.7,VFS,User,Kernel,Tenant ID(Tenant

49、 1),Atmos NFS Configuration,Atmos supports NFS v3.0Atmos exports NFS at node(tenant)level Tenant admin manages the shares Regardless of the access method,all objects are defined using global unique identifier(object ID)Translation from traditional namespace to the object ID transparentEase of share

50、configuration and management Authentication managementPolicy configuration and management based on system metadata,NFS Access,EMC Atmos v1.2.x,Share Y,Tenant ID(Tenant 2),Share A,Share B,CIFS Clients,Access Point,Access Point,Access Point,Access Point,Access Point,Access Point,Atmos Linux Driver,NFS