Open Storage Network (OSN)
Official docs (worth bookmarking!) are at https://openstoragenetwork.github.io/docs/portal/ . Note that Dartmouth owns an OSN pod so you are not an ACCESS user. You can safely ignore that section.
Ultimately OSN is simply S3 object storage with the usual concept of a bucket that has an access key (like a username) and a secret key (like the password). If you know already know how to read and write objects to one of Amazon’s S3 offerings (e.g. Glacier), using OSN is exactly the same.
OSN Concepts
-
OSN project - an organizational construct to collect “your” users separate from everyone else’s. Though people can belong to multiple projects.
-
OSN project roles - people affiliated with your project can be one of two types
manager- can see all buckets, see all bucket keys, request new buckets, add/delete users, etc.user- can see all buckets and keys, but is not able to request new buckets or manage users.
-
OSN bucket - this is where your data lives. You need at least one, but can have more if that makes sense for your work.
-
Access types. OSN buckets support 3 different types of access. You can give anyone access to a bucket by sharing the bucket name and keys with them (i.e. you do not need to add them to your OSN project)
-
read/write - one set of keys
-
read-only - second set of keys
-
anonymous read-only (no keys needed - disabled by default)
-
Accessing OSN
Using S3 storage requires you to know three things. You find this info for your OSN buckets by
- Go to https://coldfront.osn.mghpcc.org
- Click
Log In - Click
Log in with Institutional Account - [If necessary] Using the dropdown list under
Select an Identity Provider, chooseDartmouth College - From the Home page, click the green
Activebutton next to the bucket.
The three things you need are:
- The bucket endpoint which is always
https://dart1.osn.mghpcc.org - The bucket name which is specific to your bucket.
- An access key and a secret key. Safety tip: use the RO pair unless you need to be writing/changing the data.
Common methods of accessing OSN include
- Cyberduck - a graphical application for Windows and Mac. See https://openstoragenetwork.github.io/docs/dataset-access/cyberduck/
- Rclone - a nice terminal interface to the API. See https://openstoragenetwork.github.io/docs/dataset-access/rclone/
- AWS CLI - a very basic terminal interface to the API. See https://openstoragenetwork.github.io/docs/dataset-access/aws-cli/
- AWS API - the programmatic interface to S3. e.g. you can write Python code (many other languages too) and work with S3 storage directly in your code. See https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html