https://bitbucket.org/winebarrel/ddbcli
DynamoDBのmysqlコマンド的なクライアントを作りました。ほんとはAWS Tools Hackathonでこのネタをやろうとしていたのですが、DynamoDBのAPIにさわり始めたらあれよあれよと実装が進んでしまいまして。本番どうするかな…
shell> gem install ddbcli shell> export AWS_ACCESS_KEY_ID='...' shell> export AWS_SECRET_ACCESS_KEY='...' shell> export DDB_REGION=ap-northeast-1 shell> ddbcli # プロンプト表示
ddbcliコマンドを実行すると、以下のようなプロンプトが表示されます。
ap-northeast-1> show tables; [ "employees" ] // 1 row in set (0.33 sec) ap-northeast-1>
JSONをそのまま書くのは手間なので、各アクションはSQLっぽい文法で実行できるようにしています。
テーブルの作成は以下の通り。
ap-northeast-1> create table foo ( -> hoge string hash, -> fuga number range, -> index my_idx (piyo string) all) -> read = 1, write = 1;
create table foo ( hoge string hash, fuga number range, index my_idx (piyo string) all ) read = 1, write = 1;
ap-northeast-1> desc foo; { "AttributeDefinitions": [ { "AttributeName": "fuga", "AttributeType": "N" }, { "AttributeName": "hoge", "AttributeType": "S" }, { "AttributeName": "piyo", "AttributeType": "S" } ], "CreationDateTime": 1367815317.165, "ItemCount": 0, "KeySchema": [ { "AttributeName": "hoge", "KeyType": "HASH" }, { "AttributeName": "fuga", "KeyType": "RANGE" } ], "LocalSecondaryIndexes": [ { "IndexName": "my_idx", "IndexSizeBytes": 0, "ItemCount": 0, "KeySchema": [ { "AttributeName": "hoge", "KeyType": "HASH" }, { "AttributeName": "piyo", "KeyType": "RANGE" } ], "Projection": { "ProjectionType": "ALL" } } ], "ProvisionedThroughput": { "NumberOfDecreasesToday": 0, "ReadCapacityUnits": 1, "WriteCapacityUnits": 1 }, "TableName": "foo", "TableSizeBytes": 0, "TableStatus": "CREATING" } ap-northeast-1>
※TableStatusがACTIVEになるまで若干時間がかかります
「SHOW CREATE TABLE」で、CREATE文の表示も出来ます。
ap-northeast-1> show create table foo; CREATE TABLE `foo` ( `hoge` STRING HASH, `fuga` NUMBER RANGE, INDEX `my_idx` (`piyo` STRING) ALL ) read=1, write=1 ap-northeast-1>
INSERTでデータを入れます。
ap-northeast-1> insert into foo (hoge, fuga, piyo) values ('AAA', 100, 'BBB'); // 1 row changed (0.39 sec) ap-northeast-1> select all * from foo; [ {"fuga":100,"hoge":"AAA","piyo":"BBB"} ] // 1 row in set (0.08 sec) ap-northeast-1>
BULK INSERTも使えます。
ap-northeast-1> insert into foo (hoge, fuga, piyo) values ('CCC', 101, 'DDD'), ('EEE', 201, 'FFF') ; // 2 rows changed (0.44 sec) ap-northeast-1>
SELECT(Query)/SELECT ALL(Scan)でデータを取り出します。
Queryアクションは高速ですがキー属性を指定する必要があり、ちょっとデータを見るだけの時には不便です。
Scanアクションは全件走査になりますが、Queryより使えるオペレーターが多く、条件なし（フィルタなし）でもデータを取得できます。
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/QueryAndScan.html
ap-northeast-1> select * from foo where hoge = 'AAA' and fuga >= 100; [ {"fuga":100,"hoge":"AAA","piyo":"BBB"} ] // 1 row in set (0.05 sec) ap-northeast-1>
ローカルセカンダリインデックスを使う場合はUSE INDEXでインデックスを指定します。
ap-northeast-1> select * from foo use index (my_idx) where hoge = 'AAA' and piyo = 'BBB'; [ {"fuga":100,"hoge":"AAA","piyo":"BBB"} ] // 1 row in set (0.37 sec) ap-northeast-1>
デバッグモードにすると、実際に投げているHashオブジェクトが表示されます。
ap-northeast-1> .d t ap-northeast-1> select * from foo use index (my_idx) where hoge = 'AAA' and piyo = 'BBB'; ---request begin--- Action: Query {"TableName"=>"foo", "IndexName"=>"my_idx", "KeyConditions"=> {"hoge"=>{"ComparisonOperator"=>"EQ", "AttributeValueList"=>[{"S"=>"AAA"}]}, "piyo"=>{"ComparisonOperator"=>"EQ", "AttributeValueList"=>[{"S"=>"BBB"}]}}} ---request end--- ---response begin--- {"Count"=>1, "Items"=>[{"hoge"=>{"S"=>"AAA"}, "piyo"=>{"S"=>"BBB"}, "fuga"=>{"N"=>"100"}}]} ---response end--- [ {"fuga":100,"hoge":"AAA","piyo":"BBB"} ] // 1 row in set (0.05 sec) ap-northeast-1>
Scanの場合、特に条件を指定しなくてもデータを取得できます。
ap-northeast-1> select all * from foo; [ {"fuga":100,"hoge":"AAA","piyo":"BBB"}, {"fuga":201,"hoge":"EEE","piyo":"FFF"}, {"fuga":101,"hoge":"CCC","piyo":"DDD"} ] // 3 rows in set (0.68 sec) ap-northeast-1>
また、インデックスを指定しなくてもフィルタリングは可能です。
ap-northeast-1> select all * from foo where piyo = 'FFF'; [ {"fuga":201,"hoge":"EEE","piyo":"FFF"} ] // 1 row in set (0.35 sec) ap-northeast-1>
MySQLと同じように\Gで表示を変更できます。
ap-northeast-1> select all * from foo where piyo = 'FFF' \G [ { "fuga": 201, "hoge": "EEE", "piyo": "FFF" } ] // 1 row in set (0.13 sec) ap-northeast-1>
UPDATE/UPDATE ALLでデータを更新します。
UPDATEは通常のUpdateItemアクションです。キーを指定しての更新しか出来ません。
UPDATE ALLは裏でScanを行っているので、Scanでつかえるオペレータでまとめて更新できます（遅いですが）。
ap-northeast-1> update foo set xxx = 'XXX' where hoge = 'AAA' and fuga = 100; // 1 row changed (0.07 sec) ap-northeast-1> select all * from foo; [ {"fuga":100,"hoge":"AAA","piyo":"BBB","xxx":"XXX"}, {"fuga":201,"hoge":"EEE","piyo":"FFF"}, {"fuga":101,"hoge":"CCC","piyo":"DDD"} ] // 3 rows in set (0.06 sec) ap-northeast-1>
RANGEキーがある場合、HASHキーだけでは更新できないようです。SELECTはHASHキーだけでいいのに何でだろう、、、
ap-northeast-1> update all foo set zzz = 'ZZZ'; // 3 rows changed (0.49 sec) ap-northeast-1> select all * from foo; [ {"fuga":100,"hoge":"AAA","piyo":"BBB","xxx":"XXX","zzz":"ZZZ"}, {"fuga":201,"hoge":"EEE","piyo":"FFF","zzz":"ZZZ"}, {"fuga":101,"hoge":"CCC","piyo":"DDD","zzz":"ZZZ"} ] // 3 rows in set (0.07 sec) ap-northeast-1>
NULLをセットすると属性を削除できます。
ap-northeast-1> update all foo set zzz = null; // 3 rows changed (0.57 sec) ap-northeast-1> select all * from foo; [ {"fuga":100,"hoge":"AAA","piyo":"BBB","xxx":"XXX"}, {"fuga":201,"hoge":"EEE","piyo":"FFF"}, {"fuga":101,"hoge":"CCC","piyo":"DDD"} ] // 3 rows in set (0.07 sec) ap-northeast-1>
「UPDATE テーブル名 SET」の代わりに「UPDATE テーブル名 ADD」を使うと、Itemへの処理にADDを使います。
ap-northeast-1> update all foo set zzz = 100; // 3 rows changed (0.30 sec) ap-northeast-1> select all * from foo; [ {"fuga":100,"hoge":"AAA","piyo":"BBB","xxx":"XXX","zzz":100}, {"fuga":201,"hoge":"EEE","piyo":"FFF","zzz":100}, {"fuga":101,"hoge":"CCC","piyo":"DDD","zzz":100} ] // 3 rows in set (0.06 sec) ap-northeast-1> update all foo add zzz = 1000; /* set -> add */ // 3 rows changed (2.52 sec) ap-northeast-1> select all * from foo; [ {"fuga":100,"hoge":"AAA","piyo":"BBB","xxx":"XXX","zzz":1100}, {"fuga":201,"hoge":"EEE","piyo":"FFF","zzz":1100}, {"fuga":101,"hoge":"CCC","piyo":"DDD","zzz":1100} ] // 3 rows in set (0.05 sec) ap-northeast-1>
数値に数値をADDすると加算、配列に配列をADDすると配列の結合等、パターンがあるようです。
UPDATE/UPDATE ALLでデータを削除します。違いはUPDATEと同じです。
ap-northeast-1> delete from foo where hoge = 'AAA' and fuga = 100; // 1 row changed (0.27 sec) ap-northeast-1> select all * from foo; [ {"fuga":201,"hoge":"EEE","piyo":"FFF","zzz":1100}, {"fuga":101,"hoge":"CCC","piyo":"DDD","zzz":1100} ] // 2 rows in set (0.07 sec) ap-northeast-1> delete all from foo; // 2 rows changed (0.18 sec) ap-northeast-1> select all * from foo; [ ] // 0 row in set (0.05 sec) ap-northeast-1>
データが大量にある場合、反映までには時間がかかるようです。
データが一度に表示されない場合、NEXTで次のデータを表示できます。
ap-northeast-1> select all * from employees limit 3; [ {"birth_date":"1954-12-16","emp_no":35176,"first_name":"Jiafu","gender":"M","hire_date":"1998-03-05","last_name":"Wilharm"}, {"birth_date":"1960-04-16","emp_no":15886,"first_name":"Kish","gender":"M","hire_date":"1986-12-09","last_name":"Zuberek"}, {"birth_date":"1964-05-05","emp_no":13335,"first_name":"Val","gender":"F","hire_date":"1994-05-25","last_name":"Akaboshi"} ] // 3 rows in set (0.04 sec) // has more ap-northeast-1> next; [ {"birth_date":"1955-06-20","emp_no":40627,"first_name":"Rance","gender":"M","hire_date":"1992-05-28","last_name":"Hemaspaandra"}, {"birth_date":"1953-10-05","emp_no":15337,"first_name":"Masaru","gender":"M","hire_date":"1988-08-07","last_name":"Radivojevic"}, {"birth_date":"1961-02-23","emp_no":17502,"first_name":"Gor","gender":"M","hire_date":"1990-01-03","last_name":"Moehrke"} ] // 3 rows in set (0.05 sec) // has more ap-northeast-1>
末尾に「｜」をつけると、Rubyコードにデータを渡せます。
ap-northeast-1> select all * from employees limit 3 | size; 3 ap-northeast-1> select all * from employees limit 3 | emp_no.avg; 38572 ap-northeast-1> select all * from employees limit 3 | self.class; "Array" ap-northeast-1>
DynamoDBから取得したオブジェクト（たいていの場合はHashの配列）のコンテキストでコードが評価されます。
Arrayにgroup_by/avg/sumなどのメソッドを追加しています。
ap-northeast-1> select all * from employees where first_name begins_with 'Al' | group_by(:gender) {|i| i.length }; { "F": 76, "M": 107 } ap-northeast-1>
.h でヘルプが出ます。
ap-northeast-1> .h ##### Query ##### SHOW TABLES displays a table list SHOW REGIONS displays a region list SHOW CREATE TABLE table_name displays a CREATE TABLE statement CREATE TABLES table_name ( key_name {STRING|NUMBER|BINARY} HASH [, key_name {STRING|NUMBER|BINARY} RANGE] [, INDEX index1_name (attr1 {STRING|NUMBER|BINARY}) {ALL|KEYS_ONLY|INCLUDE (attr, ...)} , INDEX index2_name (attr2 {STRING|NUMBER|BINARY}) {ALL|KEYS_ONLY|INCLUDE (attr, ...)} , ...] ) READ = num, WRITE = num creates a table DROP TABLE table_name deletes a table ALTER TABLE table_name READ = num, WRITE = num updates the provisioned throughput GET {*|attrs} FROM table_name WHERE key1 = '...' AND ... gets items INSERT INTO table_name (attr1, attr2, ...) VALUES ('val1', 'val2', ...), ('val3', 'val4', ...), ... creates items UPDATE table_name {SET|ADD} attr1 = 'val1', ... WHERE key1 = '...' AND ... UPDATE ALL table_name {SET|ADD} attr1 = 'val1', ... [WHERE attr1 = '...' AND ...] [LIMIT limit] updates items DELETE FROM table_name WHERE key1 = '...' AND .. DELETE ALL FROM table_name WHERE [WHERE attr1 = '...' AND ...] [ORDER {ASC|DESC}] [LIMIT limit] deletes items SELECT {*|attrs|COUNT(*)} FROM table_name [USE INDEX (index_name)] [WHERE key1 = '...' AND ...] [ORDER {ASC|DESC}] [LIMIT limit] SELECT ALL {*|attrs|COUNT(*)} FROM table_name [WHERE attr1 = '...' AND ...] [LIMIT limit] queries using the Query/Scan action see http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/QueryAndScan.html DESC[RIBE] table_name displays information about the table USE region_or_endpoint changes an endpoint NEXT displays a continuation of a result (NEXT statement is published after SELECT statement) ##### Type ##### String 'London Bridge is...', "is broken down..." ... Number 10, 100, 0.3 ... Binary x'123456789abcd...', x"123456789abcd..." ... Identifier `ABCD...` or Non-keywords ##### Operator ##### Query (SELECT) = | <= | < | >= | > | BEGINS_WITH | BETWEEN see http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html#DDB-Query-request-KeyConditions Scan (SELECT ALL) = | <> | != | <= | < | >= | > | NOT NULL | NULL | CONTAINS | NOT CONTAINS | BEGINS_WITH | IN | BETWEEN see http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Scan.html#DDB-Scan-request-ScanFilter ##### Pass to Ruby/Shell ##### Ryby query | ruby_script ex) SELECT ALL * FROM employees WHERE gender = 'M' | birth_date.map {|i| Time.parse(i) }; [ "1957-09-16 00:00:00 +0900", "1954-12-16 00:00:00 +0900", "1964-05-23 00:00:00 +0900", ... Shell query ! shell_command ex) SELECT ALL * FROM employees LIMIT 10 ! sort; {"birth_date"=>"1957-09-16", "emp_no"=>452020,... {"birth_date"=>"1963-07-14", "emp_no"=>16998, ... {"birth_date"=>"1964-04-30", "emp_no"=>225407,... ... ##### Command ##### .help displays this message .quit | .exit exits sdbcli .consistent (true|false)? displays ConsistentRead parameter or changes it .debug (true|false)? displays a debug status or changes it .retry NUM? displays number of times of a retry or changes it .retry_interval SECOND? displays a retry interval second or changes it .timeout SECOND? displays a timeout second or changes it .version displays a version ap-northeast-1>
あまりがっつりとSimpleDBを使い込んでいる訳ではないですが、ちょっとしたデータをつっこんでおくにはなかなか便利です。特にログを残しておきたいときは特別なミドルウェアを用意する必要がないので、サーバやミドルウェアの検証をやるときにはよく使っています。
クライアントはいくつかあるのですが、コマンドラインで使えるクライアントがamazon-simpledb-cliぐらいしか見つからなかったので、一年くらい前に自作しました。
このクライアントの売りはmysqlクライアント感覚でデータを引っ張ってこれるところで、普通にSELECTはたたけるし
ap-northeast-1> select * from employees limit 3; --- - ["100000", {first_name: Hiroyasu, hire_date: "1991-07-02", birth_date: "1956-01-11", last_name: Emden}] - ["100001", {first_name: Jasminko, hire_date: "1994-12-25", birth_date: "1953-02-07", last_name: Antonakopoulos}] - ["100002", {first_name: Claudi, hire_date: "1988-02-20", birth_date: "1957-03-04", last_name: Kolinko}] # 3 rows in set
複数行のUPDATEも(一応)出来るし
ap-northeast-1> update employees set age = '35'; # 100 rows changed
と、結構便利に作っていたのですが、集計処理が出来ないのが難点でした。
一応、SQLのパーサを書いたのですが、SELECT文のパースをSimpleDBに丸投げだったりとなかなかヤクザなパーサです。集計系の構文(MAX、SUM、GROUP BY)をマジでパースしようと思うとかなり骨なので、集計処理の実装については棚上げ状態でした。
もちろん標準出力に出してRubyでこねくり回せばどうとでも出来るのですがめんどくさい！
どうしたものかと悶々としていたところ『だいたいRubyのメソッド使えばいくらでも集計できるじゃん？構文のきれいさにこだわらなければどうとでもなるんじゃね？』と思い至ったのが以下の構文です。
ap-northeast-1> select * from employees limit 3 | map {|i| i.hire_date }; --- - "1991-07-02" - "1994-12-25" - "1988-02-20" # 3 rows in set
『|』以降の文字列をRubyに丸投げというひどい代物になりました。
とはいえRubyをほぼそのまま使えるのはなかなか強力です。
first_nameだけを抽出することも出来ますし
ap-northeast-1> select * from employees limit 3 | first_name; --- - Hiroyasu - Jasminko - Claudi # 3 rows in set
first_nameが『C』で始まる人は何月生まれが多いかとかも集計できますし
ap-northeast-1> select * from employees | select {|i| i.first_name =~ /^C/ }.map {|i| Time.parse(i.birth_date).mon }.inject({}) {|r, i| r[i] ||= 0 \; r[i] += 1\; r }.sort_by {|k,v| k } ; --- - [1, 1] - [3, 1] - [5, 1] - [8, 2] - [10, 1] - [12, 3] # 6 rows in set
itemNameを数値に変換して平均を出すとか訳のわからないことも出来ます。
ap-northeast-1> select * from employees | itemname.to_f.avg; --- 91941.4
というわけで見た目はクソですが意外と便利ですので、興味のある方はどうぞご利用ください。
EC2のスポットインスタンスは安くて大変お買い得ですが、価格が上がると強制ターミネートされるのがにんともかんとも。きちんとサービスアウトできる余裕が必要なので以下のようなスクリプトを書いて終了時にsleepするようにしてみたのですが、二通目のメールは届かず…
#!/bin/bash # chkconfig: - 99 00 start() { touch /var/lock/subsys/notify-stopped } stop() { date | mail foo@example.com sleep 15 date | mail foo@example.com } restart() { stop start } case "$1" in start) start ;; stop) stop ;; restart) restart ;; *) echo "Usage: $0 {start|stop|restart}" ;; esac exit 0
いったんターミネートが始まると、さすがに速攻で殺されるようでした。
いろいろと調べてみたところ、現在の価格がMaxPriceを超えてから実際のターミネートが開始されるまで一分程度のタイムラグがあるようでした。一分も余裕があれば自分でサービスアウトするのに十分な時間です。そこで数秒おきに価格をポーリングし現在の価格がMaxPriceを超えた時に特定の処理を行うスクリプト群を書いてみました。
autognosis is a tool which processes when EC2 Spot Instance is terminated compulsorily.
スポットインスタンス激戦区のus-east-1にテスト用のインスタンスをセットアップ。スポットインスタンスの起動時には「{"maxPrice":0.3}」のようなJSON形式のuser-dataを仕込みます。
sudo yum install mailx -y wget https://bitbucket.org/winebarrel/cronexec.spec/downloads/cronexec-0.90-1.amzn1.x86_64.rpm wget https://bitbucket.org/winebarrel/jq.spec/downloads/jq-1.2-1.amzn1.x86_64.rpm wget https://bitbucket.org/winebarrel/describe-spot-price-history/downloads/describe-spot-price-history-0.1.2-1.noarch.rpm wget https://bitbucket.org/winebarrel/autognosis/downloads/autognosis-0.1.4-1.amzn1.noarch.rpm sudo rpm -ihv *.rpm sudo vi /etc/sysconfig/autognosis sudo initctl start autognosis
/etc/sysconfig/autognosisは以下の通り。
# AWS Credential AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... # expected json: {"maxPrice":0.3} MAX_PRICE=`curl -s 169.254.169.254/latest/user-data | jq '.maxPrice'` #CHECK_INTERVAL=5 ON_TERMINATE='echo detected termination: `curl 169.254.169.254/latest/meta-data/instance-id` `date` | mail -s "detected termination: $(curl 169.254.169.254/latest/meta-data/instance-id) $(date)" foo@example.com' #EXECUTE_ONCE=1 #EXEC_FLAG_FILE=/var/tmp/autognosis.executed
また、価格上昇と実際にインスタンスが停止するまでのタイムラグを計るため、終了処理開始時にメールを送るようにします。
sudo vi /etc/init.d/notify-stopped sudo chmod 755 /etc/init.d/notify-stopped sudo chkconfig --add notify-stopped sudo chkconfig notify-stopped on sudo touch /var/lock/subsys/notify-stopped #sudo /sbin/shutdown -r now
終了スクリプト（/etc/init.d/notify-stopped）は前述のものと同様。
#!/bin/bash # chkconfig: - 99 00 start() { touch /var/lock/subsys/notify-stopped } stop() { INSTANCE_ID=`curl 169.254.169.254/latest/meta-data/instance-id` DATE=`date` echo stopped: $INSTANCE_ID $DATE | mail -s "stopped: $INSTANCE_ID $DATE" foo@example.com } restart() { stop start } case "$1" in start) start ;; stop) stop ;; restart) restart ;; *) echo "Usage: $0 {start|stop|restart}" ;; esac exit 0
スポットインスタンスを立ち上げて、強制的にターミネートされるまで待った結果が以下の通り。
おおむね一分程度のタイムラグです。
us-east-1以外のリージョンではテストしていないので、もしかしたら差異があるかもしれないです。というかus-east-1以外は平和すぎてテストが長引くんですよね…。
また、公式の情報ではないので今後も一分のタイムラグが続くかは不明です。
安全な終了処理が出来ないのは結構痛いので、Amazon側でなんらかの機能を出してほしいところではあります。
#!/usr/bin/env ruby require 'cgi' require 'base64' require 'net/https' require 'openssl' require 'optparse' require 'rexml/parsers/pullparser' require 'time' require 'yaml' Net::HTTP.version_1_2 class EC2Client API_VERSION = '2012-12-01' SIGNATURE_VERSION = 2 SIGNATURE_ALGORITHM = :SHA256 def initialize(accessKeyId, secretAccessKey, endpoint = nil) @accessKeyId = accessKeyId @secretAccessKey = secretAccessKey @endpoint = endpoint if /\A[^.]+\Z/ =~ @endpoint @endpoint = "ec2.#{@endpoint}.amazonaws.com" end end def spot_price_history(params = {}) source = query('DescribeSpotPriceHistory', params) parser = REXML::Parsers::PullParser.new(source) instance_type = nil product_description = nil spot_price = nil timestamp = nil availability_zone = nil while parser.has_next? event = parser.pull next if event.event_type != :start_element case event[0] when 'instanceType' instance_type = parser.pull[0] when 'productDescription' product_description = parser.pull[0] when 'spotPrice' spot_price = parser.pull[0].to_f when 'timestamp' timestamp = Time.parse(parser.pull[0]).localtime.iso8601 when 'availabilityZone' availability_zone = parser.pull[0] yield [instance_type, product_description, spot_price, timestamp, availability_zone] end end end private def query(action, params = {}) params = { :Action => action, :Version => API_VERSION, :Timestamp => Time.now.getutc.strftime('%Y-%m-%dT%H:%M:%SZ'), :SignatureVersion => SIGNATURE_VERSION, :SignatureMethod => "Hmac#{SIGNATURE_ALGORITHM}", :AWSAccessKeyId => @accessKeyId, }.merge(params) signature = aws_sign(params) params[:Signature] = signature https = Net::HTTP.new(@endpoint, 443) https.use_ssl = true https.verify_mode = OpenSSL::SSL::VERIFY_NONE https.start do |w| req = Net::HTTP::Post.new('/', 'Host' => @endpoint, 'Content-Type' => 'application/x-www-form-urlencoded' ) req.set_form_data(params) res = w.request(req) res.body end end def aws_sign(params) params = params.sort_by {|a, b| a.to_s }.map {|k, v| "#{CGI.escape(k.to_s)}=#{CGI.escape(v.to_s)}" }.join('&') string_to_sign = "POST\n#{@endpoint}\n/\n#{params}" digest = OpenSSL::HMAC.digest(OpenSSL::Digest.const_get(SIGNATURE_ALGORITHM).new, @secretAccessKey, string_to_sign) Base64.encode64(digest).gsub("\n", '') end end # EC2Client # main access_key = nil secret_key = nil endpoint = nil params = {} ARGV.options do |opt| begin opt.on('-k', '--access-key ACCESS_KEY') {|v| access_key = v } opt.on('-s', '--secret-key SECRET_KEY') {|v| secret_key = v } opt.on('-r', '--region REGION') {|v| endpoint = v } opt.on('' , '--start-time TIME') {|v| params['StartTime'] = Time.parse(v).iso8601 } opt.on('' , '--end-time TIME') {|v| params['EndTime'] = Time.parse(v).iso8601 } opt.on('-t', '--types TYPE_LIST', Array) {|v| v.each_with_index {|t, i| params["InstanceType.#{i + 1}"] = t } } opt.on('-d', '--descs DESC_LIST', Array) {|v| v.each_with_index {|t, i| params["ProductDescription.#{i + 1}"] = t } } opt.on('-z', '--zone AVAILABILITY_ZONE') {|v| params['AvailabilityZone'] = v } opt.parse! access_key ||= ENV['AWS_ACCESS_KEY_ID'] secret_key ||= ENV['AWS_SECRET_ACCESS_KEY'] endpoint ||= (ENV['REGION_NAME'] || ENV['EC2_ENDPOINT']) unless access_key and secret_key and endpoint puts opt.help exit 1 end if (availability_zone = params['AvailabilityZone']) region = endpoint if region =~ /\Aec2\.([^.]+)\.amazonaws\.com\Z/ region = $1 end if availability_zone =~ /\A[a-z]\Z/i params['AvailabilityZone'] = region + availability_zone end end rescue => e $stderr.puts e exit 1 end end def to_yaml_style; :inline; end ec2cli = EC2Client.new(access_key, secret_key, endpoint) puts '---' ec2cli.spot_price_history(params) do |row| puts YAML.dump(row).slice(2..-1) end
~/work$ ./ec2-describe-spot-price-history -k "..." -s "..." -r us-west-1 -t t1.micro,c1.xlarge -d Linux/UNIX,Windows --start-time=10:00 --end-time=16:00 --- - [c1.xlarge, Linux/UNIX, 0.11, "2013-01-16T13:33:46+09:00", us-west-1c] - [t1.micro, Windows, 0.007, "2013-01-16T12:49:00+09:00", us-west-1b] - [c1.xlarge, Windows, 0.24, "2013-01-16T10:30:29+09:00", us-west-1c] - [c1.xlarge, Windows, 0.24, "2013-01-16T10:30:29+09:00", us-west-1a] - [t1.micro, Windows, 0.007, "2013-01-16T10:28:50+09:00", us-west-1c] - [t1.micro, Windows, 0.007, "2013-01-16T10:01:03+09:00", us-west-1a] - [c1.xlarge, Windows, 0.24, "2013-01-16T03:49:35+09:00", us-west-1b] - [t1.micro, Linux/UNIX, 0.004, "2013-01-16T02:24:39+09:00", us-west-1b] - [t1.micro, Linux/UNIX, 0.004, "2013-01-15T19:52:38+09:00", us-west-1c] - [c1.xlarge, Linux/UNIX, 0.11, "2013-01-15T19:09:04+09:00", us-west-1b] - [c1.xlarge, Linux/UNIX, 0.11, "2013-01-15T17:14:45+09:00", us-west-1a] - [t1.micro, Linux/UNIX, 0.004, "2013-01-15T17:13:26+09:00", us-west-1a] - [c1.xlarge, Linux/UNIX, 0.11, "2013-01-15T13:32:47+09:00", us-west-1c] - [t1.micro, Windows, 0.007, "2013-01-15T12:48:13+09:00", us-west-1b] - [c1.xlarge, Windows, 0.24, "2013-01-15T10:30:04+09:00", us-west-1c] - [c1.xlarge, Windows, 0.24, "2013-01-15T10:30:04+09:00", us-west-1a] - [t1.micro, Windows, 0.007, "2013-01-15T10:28:27+09:00", us-west-1c] - [t1.micro, Windows, 0.007, "2013-01-15T10:00:57+09:00", us-west-1a] ~/work$
