数据管理的研究与扩展.ppt

《数据管理的研究与扩展.ppt》由会员分享，可在线阅读，更多相关《数据管理的研究与扩展.ppt（30页珍藏版）》请在三一办公上搜索。

1、1,数据管理的研究扩展,2,提 纲,数据库人的贡献Asilomar会议【1998】Lowell会议【2003】Claremont会议【2008】当前数据库领域的研究热点清华大学数据库研究组：DBGroup,3,数据库人的贡献,RDBMS的研究与开发，与DM人不同？始于60年代：问题找人的时代成熟于80年代止步于90年代核心技术Delta研究ACM SIGMOD,VLDB,IEEE ICDE怎么办？-人找问题！,4,数据库人的贡献,大量结构化数据的分析与处理数据仓库(OLTP&OLAP)数据挖掘(统计分析、人工智能AI)：专家系统的下场？原因数据质量非结构和半结构化数据新的机会？,5,Senio

2、r Database Researcher Meeting,Senior database researchers have gathered every few years to assess the state of database research and to recommend problems and problem areas deserve additional focus.Laguna Beach,Calif.in 1989 1 Palo Alto,Calif.(“Lagunita”)in 1990 2 and 1995 3 Cambridge,Mass.in 1996 4

3、 Asilomar,Calif.in 1998 5 Lowell,Mass.In 2003 6 Claremont,Calif.in 2008 7,6,Asilomar会议：背景,在过去的十年里，由于数据库的核心技术已经十分成熟，数据库研究人员主要做的是“Delta-X”研究；但随着信息技术的迅猛发展，基本的数据库系统问题已经戏剧性地发生了变化。有一些新的丰富的问题需要数据库系统研究人员去调研；因此让研究人员离开短期的Delta研究工作而转向新的领域已经是不可逆转的时代召唤。,7,Asilomar会议：驱动力,以下三个方面的驱动形成了数据库系统研究的新焦点：Web和Internet使得把所有信

4、息放到电脑空间(Cyberspace)更加容易和具有吸引力，而且使得几乎所有的人都可访问这些信息；更加复杂的应用环境已经增加了把程序和数据集成在一起的需要；硬件的发展使得在当前DBMS技术中的假设和设计决策都荡然无效。,8,Asilomar会议：Web改变了一切,Web已成为交付应用和信息的主要平台，可利用带宽的不断增加只能加速这个进程。Web本身就是一个巨大的数据库；Web正超着管理动态内容的方向发展，而不是静态的HTML页面。支持个性化服务的下一代Web应用将需要非常高级的数据库服务；Web内容的生产商需要工具来快速而经济地建立巨大的数据商店和高级应用。这就对数据库技术创造了巨大的需求：那

5、就是Web内容的自动化创建、管理、搜索和安全保证。,9,Asilomar会议：建议的研究内容,即插即用的数据库管理系统：Sensor Network and Sensor Data联邦百万的数据库系统重新考虑传统数据库系统的体系结构智能数据库系统：数据与逻辑结构与半结构化数据的集成,10,Asilomar会议：巨大的挑战,在十年之内，大部分人类信息将会出现在Web上；它将以多种格式、E的数量级在电脑空间传播。由于缺少新的工具，发现和理解用户提出的问题的答案比今天更加困难；一个理想的系统能够简洁地回答用户的问题，能够通过通知我们一些有趣的事件来预测问题；换句话说，我们的目标就是经过十年的时间把W

6、eb变成更加有用的信息实用设施，使得每个人在存储、组织、访问和分析联机的大部分人类信息时都很容易。,11,Lowell会议,This meeting focuses on:information storage,organization,management,and access;it is driven by new applications,technology trends,new synergies with related fields,and innovation within the field itself.,12,Lowell会议：Next Generation Infras

7、tructure,Integration of Text,Data,Code,and StreamsInformation FusionSensor Data and Sensor NetworksMultimedia Queries：真正的基于内容的检索！Reasoning about Uncertain Data，包括检索PersonalizationData Mining Self AdaptationPrivacyNew User InterfacesTrustworthy Systems One-Hundred-Year Storage Query Optimization,13,C

8、laremont会议,This meeting focuses on:new database engine architectures;declarative programming languages;the interplay of structured and unstructured data;cloud data services;mobile and virtual worlds.,14,Claremont会议建议的研究机会,Revisiting Database Engines:从OLTP到Web Data,XML等类型的数据处理，多引擎？Declarative Program

9、ming for Emerging Platforms:manycore chips,distributed services,and cloud computing platforms.Non-expert programmers,15,Claremont会议建议的研究机会,The Interplay of Structured and Unstructured Data:The first challenge we face is to extract structure and meaning from unstructured and semi-structured dataThe s

10、econd challenge is to develop methods for effectively querying and deriving insight from the resulting sea of heterogeneous data,16,Claremont会议建议的研究机会,Cloud Data Services:数据中心？Application services()Storage services(Amazon S3)Compute services(Google App Engine,Amazon EC2)Data services(Amazon SimpleDB

11、,Microsoft SQL Server Data Services,Googles Datastore),17,Claremont会议建议的研究机会,Mobile Applications and Virtual Worlds:移动数据管理普适计算：context-aware的数据管理?移动终端设备：Flash-based 现实世界，还是Computers World？,18,当前数据库领域的研究热点,半结构化数据的查询处理（多引擎）XML Query Processing,IBM DB2异构数据的集成与检索：DB+IR Keyword Search over Unstructured,S

12、emi-structured and Structured Data;真正的海量数据的存储与管理（云数据服务）Google Space,Data Space,云计算新的存储介质给数据管理带来的挑战（移动特征）Flash-based OS,DBMS数据发布与挖掘中的隐私保护Data Privacy and Data Security,19,重要的参考文献：http:/db.cs.berkeley.edu/claremont/claremontreport08.pdf,20,Research Interests and Group Membershttp:/,Research InterestsK

13、eyword Search over Unstructured,Semi-structured and Structured Data；Uncertain DataXML Query ProcessingData Privacy and Data SecurityFlash-based DBMS,Group MembersGuoliang Li(PhD Student)Yu Liu(PhD Student)Jiannan Wang(PhD Student)Le Liu(Master Student)Xiaoyuan Jin(Foreign Student)Fang Xie(Master Stu

14、dent)Xiaoming Song(Master Student)Shan Jin(Foreign Student)Jing Xu(Master Student),21,Keyword Search andFuzzy Search,All Data:Unsturctured(HTML):Google 问题：是否考虑超链接？Semi-structured(XML):XQuery/LCA/SLCA：结构相似？Structured(RDB):SQL/Steiner trees：内容相似与结构相似？解决方案：统一图模型索引机制（倒排索引）：图索引DB+IR的排序机制（内容相似+结构相似）,22,异构

15、数据的关键字检索系统：EASE,用图结构来统一抽象无结构、半结构化和结构化数据求所有的r-半径图，得到包含关键字的有意义的最大子图对所有的r-半径图建立图索引利用图相似和聚类算法（K-Means）对图进行合理的划分，并维护r-半径图的中心节点文本相似和结构相似相结合的排序机制,23,原型系统：http:/166.111.69.121/EASE/index.htm,24,异构数据的关键字检索方法与技术,25,Flash-based DBMS,趋势：Magnetic Disk Flash Disk新的介质带来新的革命：OS、DBMS等主要研究F-DBMS的I/O特性,26,Current Proj

16、ects,Researches on the Key Problems of Query Processing for Native XML Databases.NSFC:60573094,2006-2008;Researches on the Storage Technology and Performance Evaluation of Flash-based DBMS.NSFC:60873065,2009-2011;Researches on the Keyword Search over Unsturctured,Semi-structured and Structured Data.

17、863 Program:2007AA01Z152,2007-2010;A Design and Implementation for Parallel XQuery Execution.Supported by the Intel Semiconductor Ltd.；Effective Schema-Aware Keyword Search over Heterogeneous Data.Supported by the HP Labs Innovation Research Program；973项目子课题；华为软件、华东电网等横向项目；核高基阳光工程,27,Research&Develo

18、pment,Scheme-aware Keyword Search;Built-in Keyword Search over RDBMS;All Features-based Search Engine for Tsinghua Web Site;MAP Search(Image=Graph)；员工产品关联的智能检索（基于DBMS和WEB的结构化数据与非结构化数据的集成与整合，HP企业要求）；核高基阳光工程,28,Selected Publications,Guoliang Li,Jianhua Feng,Jianyong Wang,Lizhu Zhou.Incremental Sequenc

19、e-Based Frequent Query Pattern Mining from XML Queries.Accepted to appear in Int.J.Data Mining and Knowledge Discovery.(DMKD,IF=2.42).Shengyue Ji,Guoliang Li,Chen Li,Jianhua Feng.Effecive Fuzzy Interactive Search.Accepted by WWW 2009.Yang Ye,Yu Zheng,Yukun Chen,Jianhua Feng,Xing Xie.Mining Individua

20、l Life Pattern Based on Location History.Accepted by MDM 2009.Guoliang Li,Xiaofang Zhou,Jianhua Feng,Jianyong Wang.Progressive Top-k Keyword Search in Relational Database.Accepted by IEEE ICDE 2009.Guoliang Li,Jianhua Feng,Beng Chin Ooi,Jianyong Wang,and Lizhu Zhou.An effective 3-in-1 keyword search

21、 method over heterogeneous data sources.Accepted to appear in Int.J.Information Systems(IF=1.681),2008.(Invited from ACM SIGMOD08 paper).Guoliang Li,Jianhua Feng,Jianyong Wang,Lizhu Zhou.An Effective and Versatile Keyword Search Engine on Heterogenous Data Sources.PVLDB 1(2):1452-1455,2008.,29,Selec

22、ted Publications,Guoliang Li,Beng Chin Ooi,Jianhua Feng,Jianyong Wang,Lizhu Zhou.EASE:Efficient and Adaptive Keyword Search on Unstructured,Semi-structured and Structured Data.ACM SIGMOD 2008:903-914.Jianhua Feng,Guoliang Li,Na Ta.A Semantic Cache Framework for Secure XML Queries.JCST,23(6):988-997,

23、November 2008.Guoliang Li,Xuhui Liu,Jianhua Feng,Jianyong Wang,Lizhu Zhou.Efficient Similarity Search for Tree-Structured Data.SSDBM 2008:131-149.Guoliang Li,Jianhua Feng,Jianyong Wang,Lizhu Zhou.Efficient Keyword Search for Valuable LCAs over XML Documents.ACM CIKM 2007:31-40.Jianhua Feng,Yuguo Lia

24、o,Yong Zhang.HCH for Checking Containment of XPath Fragment.JCST,22(5):736-748,September 2007.Jianhua Feng,Qian Qian,Jianyong Wang,Lizhu Zhou.Efficient Mining of Frequent Closed XML Query Pattern.JCST,22(5):725-735,September 2007.Jianhua Feng,Na Ta,Yong Zhang,Guoliang Li.Exploit Sequencing Views in

25、Semantic Cache to Accelerate XPath Query Evaluation.WWW 2007:1337-1338.,30,Selected Publications,Charu C.Aggarwal,Na Ta,Jianyong Wang,Jianhua Feng,Mohammed J.Zaki.XProj:A Framework for Projected Structural Clustering of XML Documents.ACM SIGKDD 2007:46-55.Jianhua Feng,Qian Qian,Jianyong Wang,Lizhu Zhou.Exploit Sequencing to Accelerate Hot XML Query Pattern Mining.ACM SAC 2006:517-524.Guoliang Li,Jianhua Feng,Jianyong Wang,Yong Zhang,Lizhu Zhou.Incremental Mining of Frequent Query Patterns from XML Queries for Caching.IEEE ICDM 2006:350-361.,