Audvik Labs

AWS Data Migration

Secure copy 

What is SCP?

Secure copy or SCP is used to transfer files from one system to another in a secure way.

Steps to create .pem file

  1. Go to AWS – EC2 Dashboard
  2. From Left Navigation click on Key Pairs : This will land you on key pairs page
  3. Click on Create Key Pair (Blue Color Button on top)
  4. Provide a unique name for the key pair. This will trigger a download of .pem file

NB: Key Pair File is created only while creating Key Pair so keep it safe.

Example:

scp -i /path/my-key-pair.pem ec2-user@ec2-198-51-100-1.compute-1.amazonaws.com:~/SampleFile.txt ~/SampleFile2.txt

Please follow the steps to store DB

Command to restore

pg_restore -v -h **url** -U **username** -p **port:5432** -d **db** database.dump

NB
You can look at working with security groups with these links

[+] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html#vpc-security-groups
[+] http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.RDSSecurityGroups.html

Once you have transferred the dumpfile and confirmed connectivity to the RDS instance, you can proceed.

At a high level the steps involved to import a PostgreSQL database into RDS involve these steps.

(1) Create a file using pg_dump that contains the data to be loaded

(2) Create the target DB instance

(3) Use psql to create the database on the DB instance and load the data

(4) Create a DB snapshot of the DB instance

The documentation below provides more granular detail regarding each step in the process and walks you through the entire migration steps.

[+]http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL.Procedural.Importing.html.

A few things to note prior to beginning the migration process into RDS are

– Ensure to disable DB instance backups (set backup_retention to 0)

– Ensure to disable Multi-AZ

Modify your DB parameter group to include the following settings.

– Increase the value of the maintenance_work_mem parameter

– Increase the value of the checkpoint_segments and checkpoint_timeout parameters to reduce the number of writes to the wal log

– Disable the synchronous_commit parameter (do not turn off FSYNC)

– Disable the PostgreSQL auto vacuum parameter

You should test the parameter settings to find the most efficient settings for your DB instance.

I hope that the information provided helps you resolve your issue.  Please feel free to reach out if you need further assistance.

Leave a comment

Your email address will not be published. Required fields are marked *