Saturday, July 7, 2018

AWS Enhanced Networking



Enhanced networking uses single root I/O virtualization (SR-IOV) to provide high-performance networking capabilities on supported instance types. SR-IOV is a method of device virtualization that provides higher I/O performance and lower CPU utilization when compared to traditional virtualized network interfaces. Enhanced networking provides higher bandwidth, higher packet per second (PPS) performance, and consistently lower inter-instance latencies. There is no additional charge for using enhanced networking.

Enhanced Networking Types

Depending on your instance type, enhanced networking can be enabled using one of the following mechanisms:
Intel 82599 Virtual Function (VF) interface
The Intel 82599 Virtual Function interface supports network speeds of up to 10 Gbps for supported instance types.
C3, C4, D2, I2, M4 (excluding m4.16xlarge), and R3 instances use the Intel 82599 VF interface for enhanced networking.
Elastic Network Adapter (ENA)
The Elastic Network Adapter (ENA) supports network speeds of up to 25 Gbps for supported instance types.
C5, C5d, F1, G3, H1, I3, m4.16xlarge, M5, M5d, P2, P3, R4, and X1 instances use the Elastic Network Adapter for enhanced networking.

AWS Data Encryption at Rest


AWS Data Encryption At Rest



Encryption on any system requires three components: 
(1) Data to Encrypt; 
(2) Encryption Method: A method to encrypt the data using a cryptographic algorithm; and 
(3) Encryption keys to be used in conjunction with the data and the algorithm. 


Security of encryption keys is often performed using a Key Management infrastructure (KMI)A KMI is composed of two sub-components: 
[1] Key Storage: The storage layer that protects the plaintext keys and 
[2] Key Management: The management layer that authorizes key usage. 

A common way to protect keys in a KMI is to use a Hardware Security Module (HSM). An HSM is a dedicated storage and data processing device that performs cryptographic operations using keys on the device. 



Model A: You control the encryption method and the entire KMI

You have full control over the encryption keys and the execution environment that utilizes those keys in the encryption code.


  • The physical location of the KMI and the encryption method can be in your data center or in an Amazon Elastic Compute Cloud (Amazon EC2) instance you own 
  •  AWS has no access to your keys and cannot perform encryption or decryption on your behalf.
Amazon S3

  • You can encrypt data using any encryption method you want, and then upload the encrypted data using the Amazon Simple Storage Service (Amazon S3) API.
  • Encrypt using 3rd party tools like Bouncy Castle and OpenSSL. 
  • Use PUT command to send the encrypted object in the S3 bucket.
  • While getting the data using GET command the object is sent through decrypting application.
AWS provides open source Amazon S3 Encryption client that encrypts and decrypts call to S3 when provided with a Key.






There are third-party solutions available that can simplify the key management process when encrypting data to Amazon S3. CloudBerry Explorer PRO for Amazon S3 and CloudBerry Backup both offer a client-side encryption option that applies a user-defined password to the encryption scheme to protect files stored on Amazon S3. 



Amazon EBS

There are two types of encryption possible with EBS.


  • Block Level Encryption: This encrypts the whole block. open source encryption solutions for Linux are Loop-AES, dm-crypt (with or without) LUKS, and TrueCrypt
These reside below operating system operating with kernel device drivers to encrpty/decrpty any file stored in the block.

  • File System Level Encryption: This encrypts spefic directory. There is open source for unix eCryptfs and EncFs for example. These solutions sit on top of the existing file system.
Both the above approach needs that you supply and manage the Keys. With these, the boot volume cannot be encrypted. 

There are open source solution that manages the keys as well as encryption of the block. Trend Micro SecureCloud and SafeNet ProtectV are two such partner products that encrypt Amazon EBS volumes and include a KMI.

Special consideration of the tools:

  • Encrypt the boot volumes
  • Attaching EBS to auto sclaed instances.

AWS Storage Gateway

AWS Storage Gateway exposes your network as an iSCSI disk to facilitate copying data from other sources. Data on disk volumes attached to the AWS Storage Gateway will be automatically uploaded to Amazon S3 based on policy. 

Encryption could be performed as one of the following:



  • You can encrypt source data on the disk volumes using any of the file encryption methods described previously (e.g., Bouncy Castle or OpenSSL) before it reaches the disk. 
  • You can also use a block- level encryption tool (e.g., BitLocker or dm-crypt/LUKS) on the iSCSI endpoint that AWS Storage Gateway exposes to encrypt all data on the disk volume. 
  • Alternatively, two AWS partner solutions, Trend Micro SecureCloud and SafeNet StorageSecure, can perform both encryption and Key Management.
Amazon RDS



Amazon RDS doesn’t expose the attached disk it uses for data storage, transparent disk encryption using techniques described in the previous Amazon EBS section are not available to you.
Selective encryption of database fields in your application can be done using any of the standard encryption libraries mentioned previously (e.g., Bouncy Castle, OpenSSL) before the data is passed to your Amazon RDS instance. While this specific field data would not easily support range queries in the database, queries based on unencrypted fields can still return useful results. The encrypted fields of the returned results can be decrypted by your local application for presentation. 

To support more efficient querying of encrypted data, you can store a keyed-hash message authentication code (HMAC) of an encrypted field in your schema and you can supply a key for the hash function. Subsequent queries of protected fields that contain the HMAC of the data being sought would not disclose the plaintext values in the query. This allows the database to perform a query against the encrypted data in your database without disclosing the plaintext values in the query. Any of the encryption methods you choose must be performed on your own application instance before data is sent to the Amazon RDS instance.

CipherCloud and Voltage Security are two AWS partners with solutions that simplify protecting the confidentiality of data in Amazon RDS. Both vendors have the ability to encrypt data using format-preserving encryption (FPE) that allows ciphertext to be inserted into the database without breaking the schema. They also support tokenization options with integrated lookup tables. 

Amazon EMR


Amazon Elastic MapReduce (Amazon EMR) provides an easy-to-use Hadoop implementation running on Amazon EC2. Performing encryption throughout the MapReduce operation involves encryption and key management at four distinct points:
  1. The source data
  2. Hadoop Distributed File System (HDFS)
  3. Shuffle phase
  4. Output data
If the source data is not encrypted, then this step can be skipped, and SSL can be used to help protect data in transit to the Amazon EMR cluster. If the source data is encrypted, then your MapReduce job will need to be able to decrypt the data as it is ingested. If your job flow uses Java and the source data is in Amazon S3, you can use any of the client decryption methods described in the previous Amazon S3 sections. 


Model B: You control the encryption method, AWS provides the KMI storage component, and you provide the KMI management layer



This model is similar to Model A in that you manage the encryption method, but it differs from Model A in that the keys are stored in an AWS CloudHSM appliance rather than in a key storage system that you manage on-premises.


Model C: AWS controls the encryption method and the entire KMI



In this model, AWS provides server-side encryption of your data, transparently managing the encryption method and the keys.


AWS KMS and other services that encrypt your data directly use a method called envelope encryption to provide a balance between performance and security. Figure 8 describes envelope encryption.
  1. A data key is generated by the AWS service at the time you request your data to be encrypted.
  2. Data key is used to encrypt your data.
  3. The data key is then encrypted with a key-encrypting key unique to the service storing your data.
  4. The encrypted data key and the encrypted data are then stored by the AWS storage service on your behalf. 

Amazon S3
There are three ways of encrypting your data in Amazon S3 using server-side encryption.
  1. Server-side encryption: You can set an API flag, or check a box in the AWS Management Console, to have data encrypted before it is written to disk in Amazon S3. Each object is encrypted with a unique data key. As an additional safeguard, this key is encrypted with a periodically rotated master key managed by Amazon S3. Amazon S3 server-side encryption uses 256-bit Advanced Encryption Standard (AES) keys for both object and master keys. This feature is offered at no additional cost beyond what you pay for using Amazon S3.
  2. Server-side encryption using customer provided keys: You can use your own encryption key while uploading an object to Amazon S3. This encryption key is used by Amazon S3 to encrypt your data using AES-256. After the object is encrypted, the encryption key you supplied is deleted from the Amazon S3 system that used it to protect your data. When you retrieve this object from Amazon S3, you must provide the same encryption key in your request. Amazon S3 verifies that the encryption key matches, decrypts the object, and returns the object to you. This feature is offered at no additional cost beyond what you pay for using Amazon S3.
  3. Server-side encryption using KMS: You can encrypt your data in Amazon S3 by defining an AWS KMS master key within your account that you want to use to encrypt the unique object key (referred to as a data key in figure 8) that will ultimately encrypt your object. When you upload your object, a request is sent to KMS to create an object key. KMS generates this object key and encrypts it using the master keythat you specified earlier; KMS then returns this encrypted object key along with the plaintext object key to Amazon S3. The Amazon S3 web server encrypts your object using the plaintext object key and stores the now encrypted object (with the encrypted object key) and deletes the plaintext object key from memory. To retrieve this encrypted object, Amazon S3 sends the encrypted object key to AWS KMS. AWS KMS decrypts the object key using the correct master key and returns the decrypted (plaintext) object key to S3. With the plaintext object key, S3 decrypts the encrypted object and returns it to you. For pricing of this option please refer to the AWS Key Management Service pricing page.
    Amazon EBS
    When creating a volume in Amazon EBS, you can choose to encrypt it using an AWS KMS master key within your account that will encrypt the unique volume key that will ultimately encrypt your EBS volume. After you make your selection, the Amazon EC2 server sends an authenticated request to AWS KMS to create a volume key. AWS KMS generates this volume key, encrypts it using the master key, and returns the plaintext volume key and the encrypted volume key to the Amazon EC2 server. The plaintext volume key is stored in memory to encrypt and decrypt all data going to and from your attached EBS volume. When the encrypted volume (or any encrypted snapshots derived from that volume) needs to be re-attached to an instance, a call is made to AWS KMS to decrypt the encrypted volume key. AWS KMS decrypts this encrypted volume key with the correct master key and returns the decrypted volume key to Amazon EC2.
    Amazon Glacier
    Before it’s written to disk, data are always automatically encrypted using 256-bit AES keys unique to the Amazon Glacier service that are stored in separate systems under AWS control. This feature is offered at no additional cost beyond what you pay for using Amazon Glacier.
    AWS Storage Gateway
    The AWS Storage Gateway transfers your data to AWS over SSL and stores data encrypted at rest in Amazon S3 or Amazon Glacier using their respective server side encryption schemes.
    Amazon EMR
    S3DistCp is an Amazon EMR feature that moves large amounts of data from Amazon S3 into HDFS, from HDFS to Amazon S3, and between Amazon S3 buckets. S3DistCp supports the ability to request Amazon S3 to use server-side encryption when it writes EMR data to an Amazon S3 bucket you manage. This feature is offered at no additional cost beyond what you pay for using Amazon S3 to store your Amazon EMR data. Oracle on Amazon RDS
    You can choose to license the Oracle Advanced Security option for Oracle on Amazon RDS to leverage the native Transparent Data Encryption (TDE) and Native Network Encryption (NNE) features. The Oracle encryption module creates data and key- encrypting keys to encrypt the database. The key-encrypting keys specific to your Oracle instance on Amazon RDS are themselves encrypted by a periodically-rotated 256-bit AES master key. This master key is unique to the Amazon RDS service and is stored in separate systems under AWS control.
    Microsoft SQL Server on Amazon RDS
    You can choose to provision Transparent Data Encryption (TDE) for Microsoft SQL Server on Amazon RDS. The SQL Server encryption module creates data and key- encrypting keys to encrypt the database. The key-encrypting keys specific to your SQL Server instance on Amazon RDS are themselves encrypted by a periodically-rotated, regional 256-bit AES master key. This master key is unique to the Amazon RDS service and is stored in separate systems under AWS control. This feature is offered at no additional cost beyond what you pay for using Microsoft SQL Server on Amazon RDS.
    Amazon Redshift
    When creating an Amazon Redshift cluster, you can optionally choose to encrypt all data in user-created tables. There are three options to choose from for server-side encryption of an Amazon Redshift cluster.
    1. In the first option, data blocks (included backups) are encrypted using random 256- bit AES keys. These keys are themselves encrypted using a random 256-bit AES database key. This database key is encrypted by a 256-bit AES cluster master key that is unique to your cluster. The cluster master key is encrypted with a periodically- rotated regional master key unique to the Amazon Redshift service that is stored in separate systems under AWS control. This feature is offered at no additional cost beyond what you pay for using Amazon Redshift.
    2. With the second option, the 256-bit AES cluster master key used to encrypt your database keys is generated in your AWS CloudHSM or by using a SafeNet Luna HSM appliance on-premises. This cluster master key is then encrypted by a master key that never leaves your HSM. When the Amazon Redshift cluster starts up, the cluster master key is decrypted in your HSM and used to decrypt the database key, which is sent to the Amazon Redshift hosts to reside only in memory for the life of the cluster. If the cluster ever restarts, the cluster master key is again retrieved from your HSM—it is never stored on disk in plaintext. This option lets you more tightly control the hierarchy and lifecycle of the keys used to encrypt your data. This feature is offered at no additional cost beyond what you pay for using Amazon Redshift (and AWS CloudHSM if you choose that option for storing keys).
    3. In the third option, the 256-bit AES cluster master key used to encrypt your database keys is generated in AWS KMS. This cluster master key is then encrypted by a master key within AWS KMS. When the Amazon Redshift cluster starts up, the cluster master key is decrypted in AWS KMS and used to decrypt the database key, which is sent to the Amazon Redshift hosts to reside only in memory for the life of the cluster. If the cluster ever restarts, the cluster master key is again retrieved from the hardened security appliance in AWS KMS—it is never stored on disk in plaintext. This option lets you define fine-grained controls over the access and usage of your master keys and audit these controls through AWS CloudTrail. For pricing of this option please refer to the AWS Key Management Service pricing page. 



Friday, July 6, 2018

AWS Storage Gateway

AWS Storage Gateway connects an on-premises software appliance with cloud-based storage to provide seamless integration with data security features between your on-premises IT environment and the AWS storage infrastructure.

There are three types of Storage Gateways provided by AWS:


File Gateway: The gateway provides access to objects in S3 as files or file share mount points. 



  • VM Image for the gateway needs to be installed and activated. EC2 AMI is also available.
  • Create and configure your file share and associate that share with your Amazon S3 bucket so that it is accessible by clients using either the NFS or SMB protocol.
  • Files written to a file share become objects in Amazon S3, with the path as the key. There is a one-to-one mapping between files and objects, and the gateway asynchronously updates the objects in Amazon S3 as you change the files. 
  • Existing objects in the bucket appear as files in the file system, and the key becomes the path.
  • Objects are encrypted with with Amazon S3–server-side encryption keys (SSE-S3). All data transfer is done through HTTPS.
  • The service optimizes data transfer between the gateway and AWS using multipart parallel uploads or byte-range downloads, to better use the available bandwidth.
  • Local cache is maintained to provide low latency access to the recently accessed data and reduce data egress charges.


Cached Volume Gateway: 

  • Amazon S3 as your primary data storage.
  • frequently accessed data locally in your storage gateway
  • You can create storage volumes up to 32 TiB in size and attach to them as iSCSI devices from your on-premises application servers.
  • Each gateway configured for cached volumes can support up to 32 volumes for a total maximum storage volume of 1,024 TiB (1 PiB).
                                         

You also allocate disks on-premises for the VM.

  • Disks for use by the gateway as cache storage
  • Disks for use by the gateway as the upload buffer – To prepare for upload to Amazon S3, your gateway also stores incoming data in a staging area, referred to as an upload buffer.
You can take incremental backups, called snapshots, of your storage volumes in Amazon S3. These point-in-time snapshots are also stored in Amazon S3 as Amazon EBS snapshots. You can initiate snapshots on a scheduled or one-time basis.

All gateway data and snapshot data for cached volumes is stored in Amazon S3 and encrypted at rest using server-side encryption (SSE).

Stored Volume Gateway: 

You also allocate disks on-premises for the VM.
  • you can store your primary data locally, while asynchronously backing up that data to AWS.
  • data is asynchronously backed up to Amazon S3 as Amazon Elastic Block Store (Amazon EBS) snapshots.
  • Stored volumes can range from 1 GiB to 16 TiB in size and must be rounded to the nearest GiB. Each gateway configured for stored volumes can support up to 32 volumes and a total volume storage of 512 TiB (0.5 PiB).
                                            


Tape Gateway: 
Refer Snowball.




AWS Enhanced Networking

Enhanced networking uses single root I/O virtualization (SR-IOV) to provide high-performance networking capabilities on  supported instanc...