danileleeopenstackblocklevelstorage120813103504phpapp01.ppt
2012/8/11,OpenStack APAC Conference 2012,1,1,Distributed Block-level Storage Management for OpenStack OpenStack APAC Conference 2012,Daniel LeeCCMA/ITRICloud Computing Center for Mobile ApplicationsIndustrial Technology Research Institute(雲端運算行動運用科技中心),2012/8/11,OpenStack APAC Conference 2012,2,Outline,OverviewBrief on ITRI Cloud OSCloud storage system in ITRI Cloud OSDistributed Main Storage SystemDistributed Secondary Storage SystemIntegration of Cloud OS Storage System with OpenStackSummaryQ&A,2012/8/11,OpenStack APAC Conference 2012,3,Overview,Data is growing exponentially and moving on to Could.It needs to be available at all time,secured and protected.911 in Twin Towers,311 earthquake in Japan and 420 outage of Amazon datacenters are disasters.Block-level and fully redundantBackup/restore+wide-areaThin provisioningDe-duplicationOthersCloud OS Storage System has these functionality and is ready to integrate to OpenStack,2012/8/11,OpenStack APAC Conference 2012,4,ITRI Cloud OS(An All-in-one IaaS Total Solution),2012/8/11,OpenStack APAC Conference 2012,5,ITRI Cloud OSService Interface,2012/8/11,OpenStack APAC Conference 2012,6,VDCM Assets(VDC,VC,VM),2012/8/11,OpenStack APAC Conference 2012,7,VDCM System Images,2012/8/11,OpenStack APAC Conference 2012,8,VDCM-Volumes,2012/8/11,OpenStack APAC Conference 2012,9,Cloud Storage System in Cloud OS,Cloud storage aims at cloud-scale data centers,and is designed to be scalable,available and low-cost.Main componentsDistributed Main Storage subsystem DMSProvide current image I/O request processingDistributed Secondary Storage subsystem DSSBlock-level incremental metadata only backup systemTake volume snapshot/restore,de-duplication engine,garbage collection and WADB engine,2012/8/11,OpenStack APAC Conference 2012,10,Key Features of Cloud Storage System from Users Perspective,Use like raw,unformatted block devices Support volume size from 1 GB to 2 TB(64TB architecture-wise)Can be attached by different instances(one at a time for now)Multiple volumes can be mounted to the same instanceData is automatically replicated on different physical storage server(N=3)Ability to clone volumes and/or create point-in-time snapshots of volumesWide-area backup for disaster recoverySnapshot can be scheduled by assigning a backup policy with time,frequency,retention,and WADB optionSave system image for starting multiple VMs using the same image,2012/8/11,OpenStack APAC Conference 2012,11,Key Features of Cloud Storage System from Providers Perspective,Use of commodity hardware reducing cost(JBOD)Disk space management for up to multiple petabytesAdd disk,remove disk without interruptionThin provisioningEnables you to provision a specific level of I/O performance if desired,by setting I/O throttleBlock-level de-duplication for reducing space requirement,2012/8/11,OpenStack APAC Conference 2012,12,DMS in Cloud OS,DMS-Distributed Main StorageGoal:A cloud scale network storage solution that provide high capacity,high reliability,high availability and high performanceFeatures:Volume operations:Create/Attach/Detach/DeleteHigh Reliability:data replication(N=3)High Availability:No single point of failureHigh Performance:client-side metadata/data cache for performanceThin provisioning is utilized to optimize utilization of available storage.Fast Volume CloningSave Image;FastBootDisk I/O Throttle,2012/8/11,OpenStack APAC Conference 2012,13,Compute Node,DMS In-kernel client,VM Volume,Service Node,Storage Node,VM Volume,VM Volume,NameNode,DataNode,Metadata,Data,DMS System Architecture,DMS three components:NamenodeDMS in-kernel clientDatanode,RR,2012/8/11,OpenStack APAC Conference 2012,14,DMS Compute Node Architecture,2012/8/11,OpenStack APAC Conference 2012,15,Compute Node,DMS In-kernel Client,VM Volume,Service Node,Storage Node,NameNode,DataNode,Read Flow,Storage Node,DataNode,Storage Node,DataNode,2012/8/11,OpenStack APAC Conference 2012,16,Compute Node,DMS In-kernel Client,VM Volume,Service Node,Storage Node,NameNode,DataNode,Write Flow Memory-Ack Approach,Storage Node,DataNode,Storage Node,DataNode,Memory-ack approach:In client,I/O response to VM(step6)is right after receiving memory ack from Datanodes(step5).,2012/8/11,OpenStack APAC Conference 2012,17,DSS in Cloud OS,Share the homogeneous space with DMSBlock-level Incremental backupCopy-on-write(COW)and volume cloning techniquesBackup is just a snapshot of meta-dataSupport volume-level restoreBlock-level de-duplicationGarbage collection of expired,un-used and redundant disk blocksWide-area data backup/restore and near-realtime replicationThe secondary storage itself is failure-tolerant(HA),2012/8/11,OpenStack APAC Conference 2012,18,DSS Backup Policy,2012/8/11,OpenStack APAC Conference 2012,19,DSS Assign Backup Policy,2012/8/11,OpenStack APAC Conference 2012,20,DSS WADB,2012/8/11,OpenStack APAC Conference 2012,21,DSS Enabling WADB,De-duplication in DropBox,22,1,2,3,DMS/DSS,1,2,3,2012/8/11,OpenStack APAC Conference 2012,2012/8/11,OpenStack APAC Conference 2012,23,DSS Software Architecture Diagram,WADB,2012/8/11,OpenStack APAC Conference 2012,24,OpenStack,Compute,Storage,Network,Identity Security,Web Interface,Object(Swift),Block,File,Hypervisors,Architectures(ARM,GPU),Switch,Router,Firewall,2012/8/11,OpenStack APAC Conference 2012,25,Integration of Cloud OS Storage System with OpenStack,User portal,Summary of Advantages,2012/8/11,OpenStack APAC Conference 2012,26,More Feature in V2.0,2012/8/11,OpenStack APAC Conference 2012,27,Q&A,Thank You!,
