How to Deploy Scalable Oracle RAC on Amazon EC2 – Part 1

This article presents one of many configurations available for deploying Oracle Real Application Clusters (RAC) on the Amazon Web Service Elastic Compute Cloud (AWS EC2).

The following procedure relies on the AWS Command Line Interface (CLI). You can run the AWS CLI on your desktop, such as a PC, Mac or Linux machine. Although all steps accomplished with the CLI can be accomplished using the AWS Console, the instructions are limited to the CLI for reasons of clarity and space.

Oracle Real Application Cluster is a shared-everything database cluster technology from Oracle. RAC allows a single database (a set of data files) to be concurrently accessed and served by one or many database server instances. Multiple instances provide high availability through redundancy at the database server level, and scaling through the addition of instances.

RAC

1- Limitations to install Oracle RAC on EC2:

1-1 In term of storage:

Shared Storage: Elastic Block Store (EBS) volumes cannot currently be attached to more than one EC2 instance at a time. This means that EBS cannot fulfill the shared storage requirement of RAC.

1-2 In term of Network Connectivity:

Multicast IP: On EC2, the network does not natively support multicast IP. However, RAC needs to broadcast packets over multicast during cluster configuration and reconfiguration events. To provide a discrete network for the interconnect, and enable it for multicast, we deploy a point-to-point VPN among the RAC nodes, using Ntop N2N.

 

2- Overcome these limitations:

To overcome this limitation, it is possible to built Amazon Machine Images (AMIs) that allow you to launch EC2 instances that can act as network attached storage (NAS) devices.

 

3- Architecture:

The selected architecture allows to build an Oracle RAC 12c cluster on EC2.

Design: The design consists of two iSCSI instances target NAS devices serving their local SSD storage, and two RAC nodes.

Performances: To minimize latency and maximize throughput, we deploy in this procedure all components on cluster compute EC2 instances with 10Gb network capabilities.

Placement Zone: To further enhance performance, this tutorial deploys all components in a single EC2 placement group, which minimizes network latencies between the instances.

Networks: To provide DNS resolution for the RAC-related network names, such as the SCAN address and the VIPs, this tutorial deploys an AWS Route 53 private hosted zone.

AWS_RAC

3-1 Obtaining the AMIs:

Availability & access: All AMIs used in this procedure are available free through AWS Marketplace.

The URLs for these products are available in:

Select the “Manual Launch” option in order to access to the AMIs in the region you choose via the AWS Command Line Interface (CLI).

Record as well the obtained AMI IDs from AWS Marketplace, as these AMI IDs will be used to launch instances in this procedure.

For example:

name: iSCSI Target Server for Oracle RAC – ami-13526564 |  (subnet 57, zone ireland west ) | storage: /dev/xvda/ | network vpc-ae83c4cb

name:  Compute Node for Oracle RAC –  ami-855f68f2 | (subnet 57, zone irelend west) | storage: /dev/sda1 | network

Cost: There is no cost for the AMIs. You only pay standard rates for EC2 usage and this is important to understand in the context of Cloud “Pay-as-you-Go”.

 

3-2 Create the user IAM and the access credentials:

PART1: NETWORK CONFIGURATION

3-3 Create VPC and Network components:

$ aws ec2 create-vpc –cidr-block 10.0.0.0/16

{

“Vpc”: {

“InstanceTenancy”: “default”,

“State”: “pending”,

“VpcId”: “vpc-8b217cee”,

“CidrBlock”: “10.0.0.0/16”,

“DhcpOptionsId”: “dopt-3e5cb65b”

}

}
Some simple add-on to enable dns support and dns hostname.

$ aws ec2 create-tags –tags Key=Name,Value=MyRacVPC –resources vpc-8b217cee
$ aws ec2 modify-vpc-attribute –vpc-id vpc-8b217cee –enable-dns-support
$ aws ec2 modify-vpc-attribute –vpc-id vpc-8b217cee –enable-dns-hostnames

if you want to be able to connect to your VPC from the Internet (Public IP), add an Internet gateway, and attach it to your VPC.

The following command creates an Internet gateway:

$aws ec2 create-internet-gateway

The following command attaches an Internet gateway to the VPC created in the previous section:

$ aws ec2 attach-internet-gateway –internet-gateway-id igw-6d5fc108 –vpc-id vpc-8b217cee

Next, determine the route table ID for your VPC using the describe-route-table-id command:

$ aws ec2 describe-route-tables –filters “Name=vpc-id,Values=vpc-8b217cee

Once the routing table is created, it is time to add a route to the VPC’s route table so that network traffic can flow through the Internet gateway. The following command adds the CIDR block 0.0.0.0/0 to the route table of the VPC:

$ aws ec2 create-route –route-table-id rtb-f36d5396 –destination-cidr-block 0.0.0.0/0 \
–gateway-id igw-6d5fc108

{

“Return”: true

}

Next you should be able to create a subnet within your VPC, and tag it with a name that you will associate with this procedure (ORACLE RAC). If you are using a pre-existing subnet, you will need to use a CIDR block appropriate to that subnet. The following commands create a subnet in the VPC, assign the CIDR block 10.0.0.0/24, and then add a tag to the subnet.

$ aws ec2 create-subnet –vpc-id vpc-8b217cee –cidr-block 10.0.0.0/24

 

{

“Subnet”: {

“VpcId”: “vpc-8b217cee”,

“CidrBlock”: “10.0.0.0/24”,

“State”: “pending”,

“AvailabilityZone”: “eu-west-1b”,

“SubnetId”: “subnet-fcc3c799”,

“AvailableIpAddressCount”: 251

}

}

Add a tag and a name to the subnet:

$ aws ec2 create-tags –resources subnet-fcc3c799 –tags Key=Name,Value=MyRacSubnet

 

3-4 Placement Policy:

In order to minimize latency between the components, create a placement group for the RAC cluster. When you deploy EC2 cluster compute instances in an EC2 placement group, it allows you to take advantage of the 10Gb Ethernet networking capabilities of these instances. The following command creates an EC2 placement group:

$ aws ec2 create-placement-group –group-name MyRACPlacementGroup –strategy cluster

 

3-5 Set up DNS:

The initial RAC cluster will have two nodes. Below is a table of the DNS names and addresses for the cluster. In this step, you will create an AWS Route 53 private hosted zone to serve these DNS names privately within your VPC. You should use addresses within your subnet’s range.

If you already use a private hosted zone for your subnet, then you don’t need to create one. The following command creates an AWS Route 53 hosted zone and associates it with the VPC. Choose a value for caller-reference that you will associate with this procedure:

active the “CreateHostedZone”

$ aws route53 create-hosted-zone –name myrachostedzone.net \
–vpc VPCRegion=eu-west-1,VPCId=vpc-8b217cee \
–caller-reference myrac-2015-12-10-a

{

“ChangeInfo”: {

“Status”: “PENDING”,

“SubmittedAt”: “2015-12-11T16:00:10.692Z”,

“Id”: “/change/C279A4ST5YNWDH”

},

“HostedZone”: {

“ResourceRecordSetCount”: 2,

“CallerReference”: “myrac-2015-12-10-a”,

“Config”: {

“PrivateZone”: true

},

“Id”: “/hostedzone/Z22USBZUADWEHH”,

“Name”: “myrachostedzone.net.”

},

“Location”: “https://route53.amazonaws.com/2013-04-01/hostedzone/Z22USBZUADWEHH”,

“VPC”: {

“VPCId”: “vpc-8b217cee”,

“VPCRegion”: “eu-west-1”

}

}

Edit a file MyRACDNSRecords to populate the DNS table and setting in the following command the HostedZone id:

aws route53 change-resource-record-sets –hosted-zone-id Z22USBZUADWEHH \
–change-batch /home/elhadi/MyRACDNSRecords.json

{

“ChangeInfo”: {

“Status”: “PENDING”,

“SubmittedAt”: “2015-12-11T16:45:56.906Z”,

“Id”: “/change/C3BBQG5ZVMBLUV”

}

}

3.6 Setting up security group and key-pair:

$ aws ec2 create-security-group –group-name MyRACSecurityGroup \
–description “My RAC Security Group” –vpc-id vpc-8b217cee

{

“GroupId”: “sg-1a2d4b7e”

}

Need authorization to access into the instances from the IP address from which you are working.

Create a key pair, so you can log in to the instances you will create. The following command creates a key pair that you will use in the next step:

$ aws ec2 create-key-pair –key-name MyRACKeyPair

 

PART2: STORAGE CONFIGURATION

3.7 Setting up shared store:

you start by creating a couple of files on our local host where you are running the AWS CLI.

Build the iSCSI target instances: The first file is a user-data script that will be used during EC2 instance creation to configure the storage servers on first boot. This file will place the SSD storage under LVM control, stripe the storage and present it via the iSCSI target service, and set the hostname. You will create one file for each of the two storage servers (01 and 02).

Using a text editor, copy the following code into a file named user-data-tgt01.sh and user-data-tgt02.sh.

——————

#!/bin/bash

ORDINAL=01
/var/opt/first_boot.sh $ORDINAL

—————–

aws ec2 run-instances –image-id ami-13526564 –instance-type i2.8xlarge –key MyRACKeyPair \
–placement AvailabilityZone=eu-west-1b,GroupName=MyRACPlacementGroup,Tenancy=dedicated \
–associate-public-ip-address –subnet subnet-fcc3c799 –private-ip-address 10.0.0.51 \
–security-group-ids sg-1a2d4b7e –user-data file://user-data-tgt01.sh

aws ec2 run-instances –image-id ami-13526564 –instance-type i2.8xlarge –key MyRACKeyPair \

–placement AvailabilityZone=eu-west-1b,GroupName=MyRACPlacementGroup,Tenancy=dedicated \
–associate-public-ip-address –subnet subnet-fcc3c799 –private-ip-address 10.0.0.52 \
–security-group-ids sg-1a2d4b7e –user-data file://user-data-tgt02.sh

{

“OwnerId”: “398958545372”,

“ReservationId”: “r-702d34dd”,

“Groups”: [],

“Instances”: [

{

“Monitoring”: {

“State”: “disabled”

},

“PublicDnsName”: “”,

“RootDeviceType”: “ebs”,

“State”: {

“Code”: 0,

“Name”: “pending”

},

“EbsOptimized”: false,

“LaunchTime”: “2015-12-14T13:27:05.000Z”,

“PrivateIpAddress”: “10.0.0.52”,

“ProductCodes”: [],

“VpcId”: “vpc-8b217cee”,

“StateTransitionReason”: “”,

    “InstanceId”: “i-fa9eb673”,

“ImageId”: “ami-13526564”,

“PrivateDnsName”: “ip-10-0-0-52.eu-west-1.compute.internal”,

“KeyName”: “MyRACKeyPair”,

“SecurityGroups”: [

{

“GroupName”: “MyRACSecurityGroup”,

“GroupId”: “sg-1a2d4b7e”

}

],

“ClientToken”: “”,

“SubnetId”: “subnet-fcc3c799”,

“InstanceType”: “i2.8xlarge”,

“NetworkInterfaces”: [

{

“Status”: “in-use”,

“MacAddress”: “02:75:6f:30:d4:b3”,

“SourceDestCheck”: true,

“VpcId”: “vpc-8b217cee”,

“Description”: “”,

“NetworkInterfaceId”: “eni-0251fe64”,

“PrivateIpAddresses”: [

{

“PrivateDnsName”: “ip-10-0-0-52.eu-west-1.compute.internal”,

“Primary”: true,

“PrivateIpAddress”: “10.0.0.52”

}

],

“PrivateDnsName”: “ip-10-0-0-52.eu-west-1.compute.internal”,

“Attachment”: {

“Status”: “attaching”,

“DeviceIndex”: 0,

“DeleteOnTermination”: true,

“AttachmentId”: “eni-attach-f54a4810”,

“AttachTime”: “2015-12-14T13:27:05.000Z”

},

“Groups”: [

{

“GroupName”: “MyRACSecurityGroup”,

“GroupId”: “sg-1a2d4b7e”

}

],

“SubnetId”: “subnet-fcc3c799”,

“OwnerId”: “398958545372”,

“PrivateIpAddress”: “10.0.0.52”

}

],

“SourceDestCheck”: true,

“Placement”: {

“Tenancy”: “dedicated”,

“GroupName”: “MyRACPlacementGroup”,

“AvailabilityZone”: “eu-west-1b”

},

“Hypervisor”: “xen”,

“BlockDeviceMappings”: [],

“Architecture”: “x86_64”,

“StateReason”: {

“Message”: “pending”,

“Code”: “pending”

},

“RootDeviceName”: “/dev/xvda”,

“VirtualizationType”: “hvm”,

“AmiLaunchIndex”: 0

}

]

}

Tag the deployed instance in the following:

$ aws ec2 create-tags –resources i-87cae20e –tags Key=Name,Value=MyiSCSITarget01

$ aws ec2 create-tags –resources i-fa9eb673 –tags Key=Name,Value=MyiSCSITarget02

 

3.8 Create the first RAC:

With the VPC, DNS and iSCSI targets in place, we are ready to launch the first RAC node:

aws ec2 run-instances –image-id ami-855f68f2 –instance-type c3.8xlarge –key MyRACKeyPair \
–placement AvailabilityZone=eu-west-1b,GroupName=MyRACPlacementGroup,Tenancy=dedicated \
–associate-public-ip-address –subnet subnet-fcc3c799 –private-ip-address 10.0.0.11 \
–security-group-ids sg-1a2d4b7e –user-data file://user-data-rac01.sh
$ aws ec2 create-tags –resources i-c1674c48 –tags Key=Name,Value=MyRACNode01

3.9 Create the second RAC:

aws ec2 run-instances –image-id ami-xxxxxxxx –instance-type c3.8xlarge –key MyRACKeyPair \
–placement AvailabilityZone=eu-west-1b,GroupName=MyRACPlacementGroup,Tenancy=dedicated \
–associate-public-ip-address –subnet subnet-fcc3c799 –private-ip-address 10.0.0.12 \
–security-group-ids sg-1a2d4b7e –user-data file://user-data-rac02.sh
$ aws ec2 create-tags –resources i-cxxxxxxxxx –tags Key=Name,Value=MyRACNode02

That’s done for the configuration and installation process for Oracle RAC in AWS using EC2 and shared storage.

About ElhadiC

Elhadi Cherkaoui has written 3 post in this blog.

Leave a Reply

%d bloggers like this: