人工智能与数据挖掘教学课件lect713.ppt

5/17/2023,AI&DM,1,Chapter 8Neural Networks,Part III:Advance Data Mining Techniques,换雍幽正洲京争窥斋敢娃嗅驮溺锁乃匠邓忿撮众刺镜恕辽拷筒跑么爬硬唬人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,2,What&Why ANN(8.1 Feed forward Neural Network)How ANN works-working principle(8.2.1 Supervised Learning)Most popular ANN-Backpropagation Network(8.5.1 The Backpropagation Algorithm:An example),Content,调顽削位浊骗帕家蚁焦糯摊恃座搓娱绩剿蛮感霜冀绕抄英门遥螟缸郴臻兜人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,3,What&Why ANN:Artificial Neural Networks(ANN),ANN is an information processing technology that emulates a biological neural network.Neuron(神经元)vs Node(Transformation)Dendrite(树突)vs InputAxon(轴突)vs OutputSynapse(神经键)vs WeightStarts in 1970s,become very popular in 1990s,because of the advancement of computer technology.,澎兴膀馋活鞘悠殴鲤椽夹约桥刽吟砌摘雾擒矗终长隶表鳖要失预友役劝裁人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,4,亨秒慑赎通垫莉言旧眩蔡跪峰渊舷漓腻祥鸥方定矩瞻龋示撼迹茧儡弱凶岸人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,5,士灌厂转乖尹尺眨歹捞复梦例诀苫秘钞在争召及珠独惕套削队疤宴韩台曼人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,6,What is ANN:Basics,Types of ANNNetwork structure,e.g.Figure 17.9&17.10(Turban,2000,version 5,p663)Number of hidden layersNumber of hidden nodesFeed forward and feed backward(time dependent problems)Links between nodes(exist or absent of links)The ultimate objectives of training:obtain a set of weights that makes all the instances in the training data predicted as correctly as possible.Back-propagation is one type of ANN which can be used for classification and estimationmulti-layer:Input layer,Hidden layer(s),Output layerFully connected Feed forwardError back-propagation,埠扎途嘲猪鼎轰僵回抚弥烙凳喜隧臃听挑羚芭密至窥斧腊循遗梦颐薪碰卯人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,7,What&Why ANN(8.1 Feed forward Neural Network)How ANN works-working principle(8.2.1 Supervised Learning)Most popular ANN-Backpropagation Network(8.5.1 The Backpropagation Algorithm:An example),Content,征胶笆室祁邱权挡惯我足究枚抱珍抑俄劳腮待木皮宪议杠豺碟恫牛祥镭邱人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,8,2.How ANN:working principle(I),Step 1:Collect dataStep 2:Separate data into training and test sets for network training and validation respectivelyStep 3:Select network structure,learning algorithm,and parametersSet the initial weights either by rules or randomlyRate of learning(pace to adjust weights)Select learning algorithm(More than a hundred learning algorithms available for various situations and configurations),注天枢羌脐班防尿竟呀升莱傻兆独砌题碴乍丛遍济障状镊喇突羹刃端汀档人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,9,2.ANN working principle(II),Step 4:Train the networkCompute outputsCompare outputs with desired targets.The difference between the outputs and the desired targets is called deltaAdjust the weights and repeat the process to minimize the delta.The objective of training is to Minimize the Delta(Error).The final result of training is a set of weights.Step 5:Test the networkUse test set:comparing test results to historical results,to find out the accuracy of the networkStep 6:Deploy developed network application if the test accuracy is acceptable,釜中烘栈都后师函搞卜陨妒莽飞赞肖嫂讹畔砧赚咨傲接埂工挣缄昭俺菩鞠人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,10,2.ANN working principle(III):Example,Example 1:OR operation(see table below)Two input elements,X1 and X2InputsCaseX1X2Desired Results1 0002011(positive)3101(positive)4111(positive),玫怪陨儒尉宏龄拉蛙孵灵村厦樱络吠涡朱宠身老痕裳镜呆馒纪蹭囱嫌抛孩人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,11,2.ANN working principle(IV):Example,Network structure:one layer(see next page)Learning algorithmWeighted sum-summation function:Y1=XiWiTransformation(transfer)function:Y1 less than threshold,Y=0;otherwise Y=1Delta=Z-YWi(final)=Wi(initial)+Alpha*Delta*XiInitial Parameters:Rate of learning:alpha=0.2Threshold=0.5;Initial weight:0.1,0.3Notes:Weights are initially random The value of learning rate-alpha,is set low first.,搓兵乌饼茁螟梦郴班肇还痘练俗转活垢尿萌挛盐怔脂管料尿蔬婉涉醚账磅人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,12,Processing Informationin an Artificial Neuron,x1,w1j,x2,Yj,w2j,Neuron j wij xi,Weights,Output,Inputs,Summations,Transfer function,赤朔状县甘挽匠季唉段乔吨妮圭怔婆盲壹笼蚀眺实狰纳帘捏佳坚默汤抓使人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,13,What&Why ANN(8.1 Feed forward Neural Network)How ANN works-working principle(8.2.1 Supervised Learning)Most popular ANN-Backpropagation Network(8.5.1 The Backpropagation Algorithm:An example),Content,恿郊舰樱砾线晰韶挫艰玲锄桶辖壕皋浩阶榷城煌旅洼赵铆副汪谐耗撇棺唯人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,14,3.Back-propagation Network,Network Topologymulti-layer:Input layer,Hidden layer(s),Output layerFully connected Feed forwardError back-propagationInitialize weights with random values,态柯搔尊亡命镐俯狗循触漳黔级术枫膛售怪杖稻令里籽短政诬粥发蜗丸革人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,15,Back-propagation Network,Output nodes,Input nodes,Hidden nodes,Output vector,Input vector:xi,wij,烷侯藏堰记诅仟骤甸泊植吱例蓝猩誓弛齿瓢弥群肆杀畅病烬刷釉胁扩钝殷人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,16,3.Back-propagation Network,For each node1.Compute the net input to the unit using summation function 2.Compute the output value using the activation function(i.e.sigmoid function)3.Compute the error4.Update the weights(and the bias)based on the error5.Terminating conditions:all wij in the previous epoch(周期)were so small as to be below some specified thresholdthe percentage of samples misclassified in the previous epoch is below some thresholda pre-specified number of epoch has expired,哇她翱俘幌轻嫩炼钥峨翅狮驮涯闻傀理娶锚冲忻舜脸背捅佛搂夕遮雇渺辊人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,17,Backpropagation Error Output Layer,簇衡吧戊菏咒资视妮勺冈帕霸豺屹耿惦筹樟肪滥洽厂餐帚逐遁苏兵侍爵州人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,18,Backpropagation Error Hidden Layer,氟谴绢券驴额找胡秘援善蚁秤稀算沼龙协民串稠攫皿妊输祖阻半畅达壁描人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,19,The Delta Rule,口傀态萝描差雹暗蘸晋桶欺眨逼秃次节麦泌性豪嫉尾唯初澡灰暗茶册肠拥人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,20,Root Mean Squared Error,手们片叹麻似伊诣胡拣玖疙欠绣片鲁俩撇肆披敦恍熊句蛹拉挺医距薪件瘟人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,21,3.Back-propagation(cont.),Increase network accuracy and training speedNetwork topologynumber of nodes in input layernumber of hidden layers(usually is one,no more than two)number of nodes in each hidden layernumber of nodes in output layerChange initial weights,learning parameter,terminating conditionTraining process:Feed the training instancesDetermine the output errorUpdate the weightsRepeat until the terminating condition is met,夹珠镶馋缆书测做营压死屡垢孝哦时轩缚首粘感膛卡亨笋莱断墅敝乓鹰砍人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,22,Supervised Learning with Feed-Forward Networks,Backpropagation Learning,讼肢绵唤遁辈骚桩篆洁刑翱环邯鼻急笛缩诞贰片币峦烬桥佩蹄糠蜜泄啦鸿人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,23,Summary:Decisions the builder must make,Network Topology:number of hidden layers,number of nodes in each layer,and feedbackLearning algorithms Parameters:initial weight,learning rateSize of training and test data,Structure and parameters determine the length oftraining time and the accuracy of the network,简搏惊瘪挚瘁料翌蛹呸迂氯邓佬槛饶液缸蓝韧距闻倾旋威精寿渝垒昨艘钮人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,24,Neural Network Input Format(Normalization:categorical to numerical),All input and output must numerical and between 0,1Categorical Attributes.e.g.attribute with 4 possible valuesOrdinal:Set to 0,0.33,0.66,1Nominal:Set to 0,0,0,1,1,0.1,1 Numerical Attributes:,替邪患肋衰怀学腥缮慰葡坦客堕贯巷典蜀虞趟氢邻坚岳姓焊未枚爸享摧俐人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,25,Neural Network Output Format,Categorical Attributes:(Numerical to categorical)Type 0&1Type 0.45Numerical Attributes:(0,1 to ordinary value)Min+X*(Max-min),禾搏利拧徒婉勃肘骑轴涝铲甘丫音靳菏攫呻振昂奶萝恩醛够隋柒揩欲炊碉人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,26,Homework,P264,Computational Questions-2 r=0.5,Tk=0.65Adjust all weights for one epoch,蛛勤瞳碴元十壁秘瑚碎师河疫乱喳碗掂泥闭沸碧矩采赎山舜怕孵钮通视御人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,27,Case Study,Example:Bankruptcy Prediction with Neural NetworksStructure:Three-layer network,back-propagationTraining data:Small set of well-known financial ratiosData available on bankruptcy outcomes Supervised network,倔奥挫陪统统肃率纵淤阂洪觅叠歇严鞘覆澜曙锡吕叁格萍除屎署确倚试虞人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,28,Architecture of the Bankruptcy Prediction Neural Network,X4,X3,X5,X1,X2,Bankrupt 0,Not bankrupt 1,焦雀禄贬哎鸭谗断炕烈膀丫蚊瓢怜返愁桌葵考浩仲滥镶趁疵疵晌澜锚恬桂人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,29,Bankruptcy Prediction:Network architecture,Five Input NodesX1:Working capital/total assets X2:Retained earnings/total assetsX3:Earnings before interest and taxes/total assetsX4:Market value of equity/total debtX5:Sales/total assetsSingle Output Node:Final classification for each firm Bankruptcy or NonbankruptcyDevelopment Tool:NeuroShell,榆范侦瓷迎黑臃悠企厂膘充费炸媒乔冬屉氛位亨谬储潜括娘媳效磐鼻窑重人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,5/17/2023,AI&DM,30,DevelopmentThree-layer network with back-error propagation(Turban,figure 17.12,p669)Continuous valued inputSingle output node:0=bankrupt,1=not bankrupt(Nonbankruptcy)TrainingData Set:129 firmsTraining Set:74 firms:38 bankrupt,36 notTestingTest data set:55 firms:27 bankrupt firms,28 nonbankrupt firmsThe neural network correctly predicted:81.5 percent bankrupt cases 82.1 percent nonbankrupt cases,岂麻末涪错忻极炔甄势乡升嫁恒怕赖王拦祈士已帖臼腿荫矛空许竿买研描人工智能与数据挖掘教学课件lect-7-13人工智能与数据挖掘教学课件lect-7-13,