Using ACL’s in Object Storage on SoftLayer

images

When you are using Object Storage in SoftLayer, there will come a time when you may find you need to share files with other SoftLayer accounts.  Unfortunately, manipulating container ACL’s appears to be not supported via the SoftLayer portal.  But, since SoftLayer Object Storage API is compatible with OpenStack Swift – this allows us to solve this problem using the API.  This means the examples in this post should work with any OpenStack Swift compatible Object Storage implementation.

I found howto’s around this topic hard to come by, so I have written up this quick guide.

First of all, ensure you have a working Python installation, and you have pip installed.

Next, install the python-swiftclient module:

$ pip install python-swiftclient

Hopefully you have a working swift command-line now like so:

$ swift
Usage: swift [--version] [--help] [--os-help] [--snet] [--verbose]
[--debug] [--info] [--quiet] [--auth <auth_url>]
[--auth-version <auth_version> |
[...]

For this example, I am using two swift configurations implemented via environment variables.  In SoftLayer, you can get your credentials from the Object Storage screen by clicking View Credentials:

view_credentials

For User A:

UserA$ cat user-a-swift-vars.sh
export ST_USER=SLO12345-2:UserA@me.com
export ST_KEY=1871e8b4595079a…
export ST_AUTH=https://syd01.objectstorage.softlayer.net/auth/v1.0/

For User B:

UserB$ cat user-b-swift-vars.sh
export ST_USER=SLO22345-2:UserB@me.com
export ST_KEY=9fe12cc1927a5877…
export ST_AUTH=https://syd01.objectstorage.softlayer.net/auth/v1.0/

Source each shell file:

UserA$ . user-a-swift-vars.sh
UserB$ . user-b-swift-vars.sh

Now, we want to share the MyNewContainer container in UserA SoftLayer account with UserB.

In the SoftLayer GUI under Object Storage the container looks like this:

20160510_093237-CapturFiles

Lets look at the default ACL’s on MyNewContainer:

UserA$ swift stat MyNewContainer
 Account: AUTH_150fef84-e459-4df7-a050-9f9f9f9f9f9c
 Container: MyNewContainer
 Objects: 2
 Bytes: 5
 Read ACL:
 Write ACL:
 Sync To:
 Sync Key:
 Accept-Ranges: bytes
X-Storage-Policy: standard
 X-Timestamp: 1462838226.47452
 X-Trans-Id: tx51d3b7ac89f64502ad3ba-0057314450
 Content-Type: text/plain; charset=utf-8

They look empty to me.  Now, lets get UserB to try and list the contents of the above object.  Note that we need to specify the URL to the storage which you can find either in the SoftLayer object storage GUI, or you can extract the important AUTH_ information from the above swift stat command. Pass –os-storage-url to swift and you can attempt to access the container:

UserB$ swift --os-storage-url https://syd01.objectstorage.softlayer.net/v1.0/AUTH_150fef84-e459-4df7-a050-9f9f9f9f9f9c list MyNewContainer
Container GET failed: https://syd01.objectstorage.softlayer.net/v1.0/AUTH_150fef84-e459-4df7-a050-9f9f9f9f9f9c/MyNewContainer?format=json 403 Forbidden [first 60 chars of response] <html><h1>Forbidden</h1><p>Access was denied to this resourc

As expected, it does not work.

Now update the ACL for MyNewContainer by adding UserB into the ACL:

UserA$ swift post MyNewContainer --read-acl "SLO22345-2:UserB@me.com"

Check that the ACL was applied:

UserA$ swift stat MyNewContainer
 Account: AUTH_150fef84-e459-4df7-a050-9f9f9f9f9f9c
 Container: MyNewContainer
 Objects: 2
 Bytes: 5
 Read ACL: SLO22345-2:UserB@me.com
 Write ACL:
 Sync To:
 Sync Key:
 Accept-Ranges: bytes
 X-Trans-Id: txfb18c6b3823c444b8e56b-005731449b
X-Storage-Policy: standard
 X-Timestamp: 1462838226.47452
 Content-Type: text/plain; charset=utf-8

Now try and list the contents of the MyNewContainer which is successful:

UserB$ swift --os-storage-url https://syd01.objectstorage.softlayer.net/v1.0/AUTH_150fef84-e459-4df7-a050-279b3d95f54c list MyNewContainer
Files
Files/test.txt

Thats it!  Some references I used for this post:

https://swiftstack.com/docs/integration/python-swiftclient.html

https://swiftstack.com/docs/cookbooks/swift_usage/container_acl.html?highlight=acl

http://docs.openstack.org/developer/swift/misc.html?highlight=acl#swift.common.middleware.acl

https://www.ibm.com/support/knowledgecenter/#!/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.adm.doc/bl1adm_createreadacl.htm

http://sldn.softlayer.com/blog/waelriac/managing-softlayer-object-storage-through-rest-apis

http://sldn.softlayer.com/reference/objectstorageapi

Advertisements

IBM SoftLayer Melbourne PoD open!

Image courtesy of http://www.computerworld.com.au/slideshow/556788/pictures-ibm-softlayer-melbourne-data-centre/
Image courtesy of http://www.computerworld.com.au/slideshow/556788/pictures-ibm-softlayer-melbourne-data-centre/

As per the press release, IBM SoftLayer Melbourne is open for business as of Tuesday 7th October 2014!

Using the SoftLayer API we can create a very short Python script to build a new virtual Ubuntu image in the new Melbourne SoftLayer datacenter. Before you begin you need a SoftLayer account and your API key which you can find under Account | Users then click on View to see your API key.

To create a virtual image you simply call the SoftLayer_Virtual_Guest::createObject API.

The parameters for this call are fairly self explanatory – with the exception of datacenter.name. How do you know what the datacenter name for Melbourne is? (without going to the customer portal that is!)

locations_api

The answer: simply call the SoftLayer_Location::getDatacenters API. For simplicity I do this via the HTTP REST API using curl, with some python json post processing to make it easy to read:

$ curl -s https://SLxxxx:API_KEY@api.softlayer.com/rest/v3/SoftLayer_Location/getDatacenters.json | python -m json.tool

An array of locations will scroll up the screen – but the important one is Melbourne:

{
"id": 449596,
"longName": "Melbourne 1",
"name": "mel01"
},

There is the magic short name – mel01.

With that information we can now create the short script to provision the guest:

import SoftLayer

client = SoftLayer.Client(username='SLxxxx', api_key='API_KEY')
client_object = client['Virtual_Guest'].createObject({
'hostname': 'test',
'domain': 'myhost.com',
'startCpus': 1,
'maxMemory': 1024,
'hourlyBillingFlag': 'true',
'operatingSystemReferenceCode': 'UBUNTU_LATEST',
"datacenter": {
"name": "mel01"
},
'localDiskFlag': 'false'
});

for key, value in client_object.iteritems():
	print key, " -> ", value

Save the file as test_build_melbourne.py.

Before you can execute it, ensure you have the python Softlayer Library installed. See https://pypi.python.org/pypi/SoftLayer for instructions.

When you are ready – give it a whirl:

$ python test_build_melbourne.py

It will sit there for a moment then return some values about the new virtual guest, then will continue executing in the background.
By using the SoftLayer sl command line interface (CLI) to the API, you can see the progress of your virtual build and find out when it is ready. Ensure you setup your sl CLI following these instructions.

provision-1

Then after a few minutes:

provision-2

After that, you can grab your root password from the password repository in the portal. You can find your passwords under Devices | Manage | Passwords:

portal_password

Your host should be listed, and just click on the password field:

portal_root_password

Side note: you can also use the API to get at your passwords:

curl 'https://SLXXXX:APT_KEY@api.softlayer.com/rest/v3/SoftLayer_Account/getVirtualGuests.json?objectMask=mask\[softwareComponents\[passwords\]\]' | python -m json.tool

Armed with your root password, you can ssh in:

$ ssh root@168.1.xxx.yyy
Password:
Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-35-generic x86_64)

* Documentation: https://help.ubuntu.com/
Last login: Tue Oct 7 06:48:35 2014 from
root@test:~#

If your ssh session is slow to respond while you are trying to login, add the following line to your /etc/ssh/sshd_config file and reboot:

UseDNS no

From my Melbourne location, the pings are nice and quick as they should be:

$ ping 168.1.xxx.yyy
PING 168.1.xxx.yyy (168.1.xxx.yyy): 56 data bytes
64 bytes from 168.1.xxx.yyy: icmp_seq=0 ttl=55 time=9.301 ms
64 bytes from 168.1.xxx.yyy: icmp_seq=1 ttl=55 time=8.006 ms
64 bytes from 168.1.xxx.yyy: icmp_seq=2 ttl=55 time=7.800 ms

If you are finished for now – you can cancel your virtual guest – as you can create a new one whenever you need it using your python script.

To cancel:

sl vs cancel 6461446

For a couple of hours work, how much will this cost me?

$ curl -s https://SLxxxxxx:API_KEY@api.softlayer.com/rest/v3/SoftLayer_Account/getBalance.xml

.04

A grand total of $US0.04 cents.

There you have it. A very quick “getting started” tutorial for creating a virtual image via three different ways of using the API (REST, CLI and Python).

The elephant in the room of cloud providers

 SaaS-PaaS-IaaS (cloud)

Of all the tech press cloud articles I read on a daily basis, almost always the comparisons are between AWS, Azure, and Google where the battle for IaaS, PaaS and SaaS market share is happening.  There is the odd exception. But, as far as I am concerned, IBM are well in this race, and definitely in the top four of cloud providers. This is confirmed by the respected Gartner IaaS magic quadrant released around 2 weeks ago which places IBM 4th behind AWS, Microsoft, and Google.  Even Google only made their debut this year with their recently released IaaS product.

I will be first to admit that IBM were a slow starter in the IaaS market. And I know this frustrated many within the company. SmartCloud Enterprise (SCE) made some progress, but it was a small player compared to the competition. It was really aimed at IBM’s existing enterprise customers, and was an attempt at providing an unmanaged cloud option to those customers who were already dipping their toes into AWS, and more recently Azure.  The problem with SCE was that it was not designed as a solution for consumers or small/mid size businesses to use easily, with the lack of a fast and easy credit card signup option.  When you only have the option for setting up infrastructure via an enterprise account manager plus purchase order, your market will be limited, and many would not call that ‘cloud’ in the first place.

The real strategy shift at IBM began with the acquisition of SoftLayer.  This provides a scale of ability to execute and agility that is competitive with the other big providers. Prior to SoftLayer, IBM were not seen as a real player in cloud. Post SoftLayer it changed the game. SoftLayer were a proven cloud provider with many years of experience (they were established the year before AWS – around 2005). As an IBMer, I have seen the changes internally and externally over the last 12 months post acquisition.  As the SoftLayer acquisition was closed in mid 2013, it was the catalyst of an internal push to educate the whole company about IBM’s clear strategy of CAMS (Cloud, Analytics, Mobile, and Social).  All employees, no matter what their role, are given training on all these areas, with a special emphasis on the first friday of the month called ThinkFriday by IBM ThinkAcademy.  This NYT article is a good read with an explanation of the strategy.

With the public beta of BlueMix, based on Cloud Foundry and built on SoftLayer, IBM started to take up the challenge to the competition.  Subsequently launched was the IBM Cloud marketplace which ties all the IBM cloud offerings together. Whether it is PaaS offerings for Devs, IaaS offerings for infrastructure needs or pre configured SaaS offerings for business solutions. Having a clear one stop shop for business needs is essential and provides a good entry point for customers to see what IBM has to offer.

IBM has made very clear its strategy to the market and will leave no stone unturned to catch up to it’s competitors with $1.2 billion being spent in the expansion of SoftLayer data centres around the world including two in Australia opening up in the next couple of months, more strategic investments (anticipated) and partnerships (e.g.: the latest with the excellent Docker OS container based virtualisation product), and continual improvement and additions to the IBM Cloud marketplace.  Out of the “big four” cloud providers, only IBM has the experience and a complete set of offerings that can cover the whole spectrum of customers, from startups and small business to high end enterprise customers with specific needs such as SAP or Oracle which they can run on SmartCloud Enterprise+ (SCE+) – the high end enterprise managed cloud offering.