php - Slow S3 -> EC2 transfer. Wrong code? -


i'm using script php5-cli download file amazon s3 ec2 instance, it's rather slow (1 minute 160 mb, 2 mb/s). supposedly @ least 10 mb/s should possible. doing wrong in code?

require 'aws-sdk/sdk.class.php';  $s3 = new amazons3(array(         'key' => '********',         'secret' => '******************' )); $s3->get_object('mahbucket', 'filename.tar.gz', array('filedownload' => 'downloaded.tar.gz'));

the bucket in eu-west-1 , instance in eu-west-1a

(cross posted aws forums)

your script looks fine, ran similar issues while ago, wasn't abled fix …

my new approach using s3cmd file transfer between s3 buckets , ec2 instances. have configure once , you're abled trigger via php. it's more secure because don't have store aws credentials inside script.

in opinion speed okay:

user@mothership:~/s3# s3cmd put test.bin s3://data.example.com/test.bin  test.bin -> s3://data.example.com/test.bin  [part 1 of 13, 15mb]  15728640 of 15728640   100% in    0s    16.39 mb/s  done test.bin -> s3://data.example.com/test.bin  [part 2 of 13, 15mb]  15728640 of 15728640   100% in    0s    15.55 mb/s  done test.bin -> s3://data.example.com/test.bin  [part 3 of 13, 15mb]  15728640 of 15728640   100% in    0s    16.18 mb/s  done test.bin -> s3://data.example.com/test.bin  [part 4 of 13, 15mb]  15728640 of 15728640   100% in    0s    17.32 mb/s  done test.bin -> s3://data.example.com/test.bin  [part 5 of 13, 15mb]  15728640 of 15728640   100% in    0s    18.87 mb/s  done test.bin -> s3://data.example.com/test.bin  [part 6 of 13, 15mb]  15728640 of 15728640   100% in    0s    16.58 mb/s  done test.bin -> s3://data.example.com/test.bin  [part 7 of 13, 15mb]  15728640 of 15728640   100% in    0s    16.29 mb/s  done test.bin -> s3://data.example.com/test.bin  [part 8 of 13, 15mb]  15728640 of 15728640   100% in    0s    16.95 mb/s  done test.bin -> s3://data.example.com/test.bin  [part 9 of 13, 15mb]  15728640 of 15728640   100% in    0s    15.33 mb/s  done test.bin -> s3://data.example.com/test.bin  [part 10 of 13, 15mb]  15728640 of 15728640   100% in    0s    17.01 mb/s  done test.bin -> s3://data.example.com/test.bin  [part 11 of 13, 15mb]  15728640 of 15728640   100% in    0s    15.36 mb/s  done test.bin -> s3://data.example.com/test.bin  [part 12 of 13, 15mb]  15728640 of 15728640   100% in    0s    16.56 mb/s  done test.bin -> s3://data.example.com/test.bin  [part 13 of 13, 10mb]  11256320 of 11256320   100% in    0s    14.15 mb/s  done  user@mothership:~/s3# s3cmd s3://data.example.com/test.bin test.bin.new s3://data.example.com/test.bin -> test.bin.new  [1 of 1]  200000000 of 200000000   100% in   11s    16.20 mb/s  done

maybe article interesting well: network performance within amazon ec2 , amazon s3


Comments

Post a Comment

Popular posts from this blog

java - Play! framework 2.0: How to display multiple image? -

i need display gallery of photos. here template: @(photos: list[photo]) @title = { <bold>gallery</bold> } @main(title,"photo"){ <ul class="thumbnails"> @for(photo <- photos) { <li class="span3"> <a href="#" class="thumbnail"> <img src="@photo.path" alt=""> </a> </li> } </ul> } and here controller method: public static result getphotos() { return ok(views.html.photo.gallery.render(photo.get())); } and here photo bean : @entity public class photo extends model { @id public long id; @required public string label; public string path; public photo(string path, string label) { this.path = path; this.label = label; } private static finder<long, photo> find = new finder<long, photo>( long.class, photo.class); public static list<phot...
Read more

gmail - Is there any documentation for read-only access to the Google Contacts API? -

i want read-only access user's google contacts. there stackoverflow member wanted this, , received following answer : yes, there is, use https://www.googleapis.com/auth/contacts.readonly scope , "view contacts". this exchange reference read-only access google contacts can find using google. simply using access token obtained attempt read usual google contacts api (i.e. https://www.google.com/m8/feeds ) fails insufficient permissions. similarly, using google contacts api on new target fails 404 errors (for example https://www.googleapis.com/auth/contacts.readonly/default/full ). is there documentation anywhere on how use api, or permissible way access google contacts api full read-write access?
Read more

php - Controller/JToolBar not working in Joomla 2.5 -

relevant code: jtoolbarhelper::custom('savecategories', 'save', '', 'save', false, false); ... <input type="hidden" name="controller" value="easyblogcontroller"> according can dig docs , own previous questions, should call savecategories() function in easyblogcontroller . i've tried setting value easyblog , easyblog.php (file name) current easyblogcontroller (class name). clicking on save button refreshes page. not redirect, echo or var_dump test code put in savecategories() function. var_dump ing jrequest::getvar('controller') , 'task' return correct values. creating controller object , using $controller->execute('task'); works. the values going depend on location of controller trying call, , have couple options here. typically there controller in base folder of component (likely components/com_easyblog) in file called controller.php , class name inside ...
Read more