杩戞棩锛涓滄柟鍥戒俊CirroData-TimeS鏃跺簭鏁版嵁搴杩涜浜嗗叏鏂板崌绾э紝鎺ㄥ嚭浜V1.3.1鐗堟湰銆傛柊鐗堟椂搴忔暟鎹簱鏂板鍥涢」鏂板姛鑳斤紝鍒嗗埆鍦瀹夊叏鎺у埗銆鏁版嵁鍚屾銆绯荤粺鐩戞帶鍜搴忓垪瑙嗗浘鏂归潰甯︾粰鐢ㄦ埛鍏ㄦ柊浣撻獙锛
姝ゅCirroData-TimeS V1.3.1涔熷鍘熸湁绯荤粺闂杩涜浜嗕慨澶嶏紝鍚屾椂浼樺寲浜嗗椤瑰姛鑳芥搷浣滐紝涓嶆柇婊¤冻鐢ㄦ埛鍦ㄦ柊鍦烘櫙涓殑鏂伴渶姹傘€
鏂板鍔熻兘
瀹夊叏鎺у埗锛氶€氳繃瀹¤鏃ュ織锛屾柟渚跨鐞嗗憳鏌ヨ鐢ㄦ埛鍦ㄦ暟鎹簱涓殑澧炲垹鏀规煡绛夊悇椤规搷浣滐紝纭繚淇℃伅瀹夊叏锛
鏂板瀹¤鏃ュ織鍔熻兘锛屽彲浠ュ疄鐜板涓嬪満鏅渶姹傦細
- 鍙互鎸夐摼鎺ユ潵婧愬喅瀹氭槸鍚﹁褰曞璁℃棩蹇楋紝渚嬪锛氶潪浜轰负鎿嶄綔锛屽儚纭欢閲囬泦鍣ㄥ啓鍏ョ殑鏁版嵁涓嶉渶瑕佽鍏ュ璁℃棩蹇楋紱浜轰负鎿嶄綔锛岃濡傛櫘閫氱敤鎴烽€氳繃cli绛夊伐鍏锋搷浣滅殑鏁版嵁锛岄渶瑕佽鍏ュ璁℃棩蹇楋紱
- 杩囨护鎺夌郴缁熺骇鍒殑鍐欏叆鎿嶄綔锛屽CirroData-TimeS鐩戞帶浣撶郴鏈韩璁板綍鐨勫啓鍏ユ搷浣滅瓑銆
鏁版嵁鍚屾锛氭弧瓒崇敤鎴锋惌寤哄畬鏁寸殑鏁版嵁閾捐矾鏉ユ弧瓒崇杈逛簯鍚屾銆佸紓鍦扮伨澶囥€佽鍐欒礋杞藉垎搴撶瓑闇€姹傦紱
鏁版嵁鍚屾鍔熻兘鍙互灏咰irroData-TimeS鐨勬暟鎹悓姝ヤ紶杈撳埌鍙︿竴涓暟鎹钩鍙帮紝鎴戜滑灏嗕竴涓暟鎹悓姝ヤ换鍔＄О涓篜ipe锛涓€涓狿ipe鍖呭惈涓変釜瀛愪换鍔★紙鎻掍欢锛夛細鎶藉彇锛圫ource锛銆澶勭悊锛圥rocess锛銆鍙戦€侊紙Sink锛銆
Pipe鍏佽鐢ㄦ埛鑷畾涔変笁涓瓙浠诲姟鐨勫鐞嗛€昏緫锛岄€氳繃绫讳技UDF鐨勬柟寮忓鐞嗘暟鎹€鍦ㄤ竴涓 Pipe 涓紝涓婅堪鐨勫瓙浠诲姟鍒嗗埆鐢变笁绉嶆彃浠舵墽琛屽疄鐜帮紝鏁版嵁浼氫緷娆＄粡杩囪繖涓変釜鎻掍欢杩涜澶勭悊锛歅ipe Source 鐢ㄤ簬鎶藉彇鏁版嵁锛孭ipe Processor 鐢ㄤ簬澶勭悊鏁版嵁锛孭ipe Sink 鐢ㄤ簬鍙戦€佹暟鎹紝鏈€缁堟暟鎹皢琚彂鑷冲閮ㄧ郴缁熴€
Pipe 浠诲姟妯″瀷濡備笅锛
鎻忚堪涓€涓暟鎹悓姝ヤ换鍔★紝鏈川灏辨槸鎻忚堪 Pipe Source銆丳ipe Processor 鍜 Pipe Sink 鎻掍欢鐨勫睘鎬с€傜敤鎴峰彲浠ラ€氳繃 SQL 璇彞澹版槑寮忓湴閰嶇疆涓変釜瀛愪换鍔＄殑鍏蜂綋灞炴€э紝閫氳繃缁勫悎涓嶅悓鐨勫睘鎬э紝瀹炵幇鐏垫椿鐨勬暟鎹 ETL 鑳藉姏銆
鍏ㄦ柊鐩戞帶闈㈡澘锛氭柊鐗圕irroData-TimeS鐨勭洃鎺ч潰鏉垮寘鎷簡闆嗙兢鐩戞帶銆佹暟鎹啓鍏ャ€佹暟鎹煡璇€佸瓨鍌ㄥ紩鎿庡拰绯荤粺鐩戞帶绛夊姛鑳斤紱
- 闆嗙兢鐩戞帶锛鐩戞帶闆嗙兢鎬籆PU鏍告暟銆佹€诲唴瀛樼┖闂淬€佹€荤‖鐩樼┖闂淬€侀泦缇ゅ寘鍚獵onfigNode涓嶥ataNode鏁般€侀泦缇ゅ惎鍔ㄦ椂闀裤€佸啓鍏ラ€熷害銆侀泦缇ゅ悇鑺傜偣褰撳墠CPU銆佸唴瀛樸€佺鐩樹娇鐢ㄧ巼銆佸垎鑺傜偣淇℃伅绛夛紱
- 鏁版嵁鍐欏叆锛鐩戞帶鍐欏叆骞冲潎鑰楁椂銆佽€楁椂涓綅鏁般€99%鍒嗕綅鑰楁椂銆乄AL鏂囦欢鏁伴噺涓庡ぇ灏忋€佽妭鐐筗AL flush SyncBuffer鑰楁椂 绛夛紱
- 鏁版嵁鏌ヨ锛鐩戞帶鑺傜偣鏌ヨ鍔犺浇鏃堕棿搴忓垪鍏冩暟鎹€楁椂銆佽鍙栨椂闂村簭鍒楄€楁椂銆佷慨鏀规椂闂村簭鍒楀厓鏁版嵁鑰楁椂銆佸姞杞紺hunk鍏冩暟鎹垪琛ㄨ€楁椂銆佷慨鏀笴hunk鍏冩暟鎹€楁椂銆佹寜鐓hunk鍏冩暟鎹繃婊よ€楁椂銆佹瀯閫燙hunk Reader鑰楁椂鐨勫钩鍧囧€ 绛夛紱
- 瀛樺偍寮曟搸锛鐩戞帶鍒嗙被鍨嬬殑鏂囦欢鏁伴噺鍜屽ぇ灏忋€佸浜庡悇闃舵鐨凾sFile鏁伴噺鍜屽ぇ灏忋€佸悇绫讳换鍔＄殑鏁伴噺涓庤€楁椂 绛夛紱
- 绯荤粺鐩戞帶锛鐩戞帶绯荤粺鍐呭瓨銆佷氦鎹㈠唴瀛樸€佽繘绋嬪唴瀛樸€佺鐩樼┖闂淬€佹枃浠舵暟銆佹枃浠跺昂瀵搞€丣VM GC鏃堕棿鍗犳瘮銆佸垎绫诲瀷鐨凣C娆℃暟銆丟C鏁版嵁閲忋€佸悇骞翠唬鐨勫爢鍐呭瓨鍗犵敤銆佺綉缁滀紶杈撻€熺巼銆佸寘鍙戦€侀€熺巼 绛夈€
搴忓垪瑙嗗浘锛氫竴绉嶅叏鏂扮殑鍩轰簬鏃堕棿搴忓垪鐨勬暟鎹粍缁囩鐞嗘柟寮忥紱
鍦ㄤ紶缁熷叧绯诲瀷鏁版嵁搴撲腑锛屾暟鎹兘蹇呴』瀛樻斁鍦ㄤ竴涓〃涓紝鑰屽湪CirroData-Times绛夋椂搴忔暟鎹簱涓紝搴忓垪鎵嶆槸瀛樺偍鍗曞厓銆傚洜姝わ紝CirroData-Times涓簭鍒楄鍥剧殑姒傚康涔熸槸寤虹珛鍦ㄥ簭鍒椾笂鐨勩€
涓€涓簭鍒楄鍥惧氨鏄竴鏉¤櫄鎷熺殑鏃堕棿搴忓垪锛屾瘡鏉¤櫄鎷熺殑鏃堕棿搴忓垪閮藉儚鏄竴鏉¤蒋閾炬帴鎴栧揩鎹锋柟寮忥紝鏄犲皠鍒版煇涓鍥惧閮ㄧ殑搴忓垪鎴栬€呮煇绉嶈绠楅€昏緫銆傛崲瑷€涔嬶紝涓€涓櫄鎷熷簭鍒楄涔堟槧灏勫埌鏌愪釜纭畾鐨勫閮ㄥ簭鍒楋紝瑕佷箞鐢卞涓閮ㄥ簭鍒楄繍绠楀緱鏉ャ€
鐢ㄦ埛鍙互浣跨敤澶嶆潅鐨凷QL鏌ヨ鍒涘缓瑙嗗浘锛屾鏃跺簭鍒楄鍥惧氨鍍忎竴鏉¤瀛樺偍鐨勬煡璇㈣鍙ワ紝褰撲粠瑙嗗浘涓鍙栨暟鎹椂锛屽氨鎶婅瀛樺偍鐨勬煡璇㈣鍙ヤ綔涓烘暟鎹潵婧愶紝鏀惧湪FROM瀛愬彞涓€
搴忓垪瑙嗗浘閫氬父鍦ㄤ互涓 瀹為檯鍦烘櫙 涓簲鐢細
- 鏃堕棿搴忓垪閲嶅懡鍚嶏紙PI璧勪骇绠＄悊锛夛細瀹為檯搴旂敤涓紝閲囬泦鏁版嵁鐨勮澶囧彲鑳戒娇鐢ㄤ汉绫婚毦浠ョ悊瑙ｇ殑鏍囪瘑鍙锋潵鍛藉悕锛岃繖缁欎笟鍔″眰甯︽潵浜嗘煡璇笂鐨勫洶闅俱€傝€屽簭鍒楄鍥捐兘澶熼噸鏂扮粍缁囩鐞嗚繖浜涘簭鍒楋紝鍦ㄤ笉鏀瑰彉鍘熸湁搴忓垪鍐呭銆佹棤闇€鏂板缓鎴栨嫹璐濆簭鍒楃殑鎯呭喌涓嬶紝浣跨敤鏂扮殑妯″瀷缁撴瀯鏉ヨ闂粬浠紱
- 绠€鍖栦笟鍔″眰鏌ヨ閫昏緫锛鏈夋椂鐢ㄦ埛鏈夊ぇ閲忚澶囷紝绠＄悊鐫€澶ч噺鏃堕棿搴忓垪銆傚湪杩涜鏌愰」涓氬姟鏃讹紝鐢ㄦ埛甯屾湜浠呭鐞嗗叾涓殑閮ㄥ垎搴忓垪锛屾鏃跺氨鍙互閫氳繃搴忓垪瑙嗗浘鍔熻兘鎸戦€夊嚭鍏虫敞閲嶇偣锛屾柟渚垮弽澶嶆煡璇€佸啓鍏ワ紱
- 杈呭姪鏉冮檺绠＄悊锛鐢熶骇杩囩▼涓紝涓嶅悓涓氬姟璐熻矗鐨勮寖鍥翠竴鑸笉鍚岋紝鍑轰簬瀹夊叏鑰冭檻寰€寰€闇€瑕侀€氳繃鏉冮檺绠＄悊鏉ラ檺鍒朵笟鍔″憳鐨勮闂寖鍥淬€
鏀硅繘涓庝紭鍖
CirroData-TimeS V1.3.1鏂扮増浼樺寲浜嗗涓嬪姛鑳芥搷浣滐細
- 浼樺寲鏉冮檺妯″潡锛屾敮鎸佹椂闂村簭鍒楃矑搴︾殑鏉冮檺鎺у埗锛
- 瀹㈡埛绔湇鍔″櫒鏀寔 SSL 閫氳鍔犲瘑锛
- 璁＄畻绫诲瀷瑙嗗浘鏀寔 LAST 鏌ヨ锛
- 浼樺寲 Python 瀹㈡埛绔殑鎬ц兘锛
- 浼樺寲鏌ヨ鎵ц杩囩▼锛
- 澧炲姞涓€閿惎鍋滈泦缇よ剼鏈紙start-all/stop-all.sh & start-all/stop-all.bat锛夛紱
- 澧炲姞涓€閿敹闆嗗疄渚嬩俊鎭剼鏈紙collect-info.sh & collect-info.bat锛夛紱
- 鏂板鏍囧噯宸€佹柟宸唴缃仛鍚堝嚱鏁帮紱
- 鏂板 TsFile 淇鍛戒护锛
- Load TsFile 瀹屽杽鍐呭瓨鎺у埗锛岃鐩栧叏娴佺▼锛
- Rest 鎺ュ彛锛圴2 鐗堬級澧炲姞鍒楃被鍨嬭繑鍥炪€
CirroData-TimeS V1.3.1鏂扮増淇浜嗕互涓嬮棶棰橈細
- 淇鍦 target path 涓紝浣跨敤鍙嶅紩鍙峰鑷 select into 鎶ラ敊鐨勯棶棰橈紱
- 淇缃戠粶鍒嗗尯鐜涓 show cluster 鍑虹幇 NPE 鐨勯棶棰橈紱
- 淇 duration 鍚 unit 鏃 group by month 寮傚父闂锛
- 淇鏈 order by clause 鏃讹紝limit 鍜 offset 涓嶈兘涓嬫帹鐨勯棶棰橈紱
- 淇 group by month + align by device + limit 缁勫悎鍦烘櫙涓嬬殑寮傚父锛
- 淇 IoT 鍗忚鍚屾鏃跺弽搴忓垪鍖栨姤閿欓棶棰樸€
浜哄伐鏅鸿兘鎶€鏈綉 鍊″灏婇噸涓庝繚鎶ょ煡璇嗕骇鏉冦€傚鍙戠幇鏈珯鏂囩珷瀛樺湪鐗堟潈绛夐棶棰橈紝鐑﹁30澶╁唴鎻愪緵鐗堟潈鐤戦棶銆佽韩浠借瘉鏄庛€佺増鏉冭瘉鏄庛€佽仈绯绘柟寮忕瓑鍙戦偖浠惰嚦1851688011@qq.com鎴戜滑灏嗗強鏃舵矡閫氫笌澶勭悊銆傦紒锛棣栭〉 > 澶ф暟鎹 » 涓滄柟鍥戒俊CirroData-TimeS鏃跺簭鏁版嵁搴撳叏鏂板崌绾э紝V1.3.1鐗堟帹鍑哄洓澶ф柊鍔熻兘