Azure Archive (Blob) Storage Service

Contents

Basics

Operations and Terms

Prior to using Azure Blob Storage, please review these Operations and Terms:

Storage account - The account that you will use to manage your Azure archival storage. This account is created for you when you purchase Azure archive storage.

Resource Group - A group that is used as a container for your Azure archival resources.

Blob - Binary Large Object. Blob storage can include files and unstructured data. 

Tier - Azure Blob Storage is provided in service tiers. The more “hot” or readily available the storage, the more expensive. The archive tier is the cheapest, but files in the archive tier must be copied to the active tier before you can read or download them.

Rehydrate - The process of moving a file from archive tier to an active tier. Files can be rehydrated by copying them to the active tier or by changing the tier level where the file is stored. Changing the tier may incur a penalty. Rehydration may take a notable amount of time.

Archive Early Deletion Penalty - Once a file is added to the archive tier, it will incur 180 days worth of cost. If the data is deleted or removed early, you must still pay the 180 days worth of fees. Copying to an active tier does not incur this penalty.

Upload - Moves data from local storage to the Azure Container.

Download - Moves data from Azure container to local storage.

Copy - Makes a copy of the file. Specifically, it is useful for rehydrating an archived file to an active tier.

Delete - File deletion takes place immediately. The blob will be recoverable for 7 days. After that, there is no way to restore the file.

Using Azure Archive Storage 

Connecting to your Azure Storage account

You can connect to your Azure Storage account through the desktop client, Azure Storage Explorer, or by using the Azure Web interface. Some functionality will vary depending on which option you used to connect to your storage account. 

Azure Web Portal

1. To log on to the Azure Web Portal, use a browser to navigate to https://portal.azure.com.
2. In the panel on the left, select Storage Accounts. Note that you may need to click the hamburger menu (horizontal bars) before you see the panel. 

3. The list of storage accounts you have access appears. Select the storage account you need, and then select Storage Browser, which will  populate after a few moments. You will then be able to go into the specific container and perform operations.

Azure Storage Explorer

Azure Storage Explorer is a free tool to work with Azure Storage accounts. It is available for Windows, macOS, and Linux from Microsoft at https://azure.microsoft.com/en-us/products/storage/storage-explorer.

To sign in to Azure Storage Explorer with your UD account, complete these steps:

If you’ve been given a SAS url, you should instead use the directions to open a storage container after you've been granted token access.

1. Once Azure Explorer has been downloaded and installed, start the program.
2. In the “Getting started” window, select Sign in with Azure.
3. Choose Azure, and click Next.
4. On the web-based login screen, sign in with your UD credentials. Once signed in, close the browser window. 
5. Click Open Explorer.

6. The storage browser window opens. You can then open your storage account and containers from Explorer so that you can perform specific operations. 

Contents

Following recommended practices

Container Setup

• Project Falcon Archive 2023
• Acronym Conference June 2019
• Investigation Data 2015

• Bob’s Desktop
• Bob’s Desktop (2)
• Bob’s Desktop (Backup - 3)
• Archive 1
• Archive 2
• Backup

Archiving and Deletion Policies

• In the Azure archive service request form [link] we ask that you specify an archiving policy for your archiving container, to make sure that the correct data is archived. We recommend that your policy be set to move things from “Cool” to “Archive” tiers after 10 days. This should provide sufficient time to ensure the correct data was uploaded before it changes to archive status. The time can be shorter if you want.
• Do not place an archive policy on your “Active” container because data that you rehydrate to that location will be archived before you have an opportunity to download it.
• Place deletion policies when appropriate, and remember they are there. There will not be a warning before the data is deleted.

Uploading data

Note: Do not overwrite data in an archive tier, because doing so will incur additional unintended/duplicated costs.

In general, Azure Storage Explorer is a robust file tool that works well even when uploading large data sets. However, these operations should be performed from a stable network connection. There are other options for mass uploading of data:

• Introduction to Azure Storage Mover | Microsoft Learn - This method runs on a resource (vm, or physical machine) close to the stored data. Jobs are run from a cloud management plane to migrate the data. The data must be accessible over NFS. This option requires central IT involvement. Please contact askit@udel.edu if you are interested in this method.
• Rclone - this tool can be run by end users. Data is copied from the source through the system running rclone, up to the target. Depending on network connections, this tool can operate very fast.

Rehydrating data

You’ll need to rehydrate archived data before you will be able to work with it in any capacity. To rehydrate data:

A. Prepare for data recovery:

1. Find the data you want to recover. Once located, note the size.
2. Have a local storage location prepared, such as your local pc or some larger storage environment. The recovery space must have enough space to hold the recovered data.

B. Rehydrate your data:

This involves copying the data from the archive tier to the active container.

1. Using Azure Data Explorer find the file you wish to recover.
2. Right-click on the file.
3. Select Clone and Rehydrate from the drop-down menu.
   1. Fill in the Destination blob container as the “Active” container. We recommend doing a standard priority rehydrate to a cool tier to keep costs reasonable.
   2. Click Apply.

3. The rehydration job is submitted. Rehydration time can vary depending on the size of the file and the overall performance at that moment. No notification is generated. Periodically check to see whether the file is in its destination. During rehydration, the access tier will be listed as “Archive (rehydrating to cool)”. This will change to “Cool” when completed.

C. Download the data, and delete the copy from the active container:

1. Use the web portal or Azure Data Explorer to download the data to local storage.
2. Once the download is complete, and you have verified the data is stored locally, delete the copy in the active container to avoid unnecessary charges.

Granting permissions

Permissions can be granted to users on a permanent (until removal) basis, or granted via a temporary token. Either process is done via the Azure Portal.

Granting User Access to Storage Container:

1. From the Azure Portal, click Storage accounts, and then click the name of your storage account.
2. Click Containers from the Data storage section on the menu panel , and click the name of the container you want to manage.
3. Select Access Control (IAM).
4. Under Role Assignments, you can see who currently has access. Several administrative accounts will appear here; this is normal.
5. On the toolbar, click Add.
6. Click Add Role assignment.
7. Generally, you should grant the least privilege that allows the accessor to complete necessary work.
   1. Storage Blob Data Owner - can read, write, delete from the container. This role is able to delete files.
   2. Storage Blob Data Contributor - contributor plus can change permissions.
   3. Storage Blob Data Reader - can only read blobs.
   4. Storage Blob Delegator - can generate access keys.
8. Select the role, and click Next. Choose user, group or service principal. Click Select members. Search for the person to add by typing in their email (fastest method) or their full name. 
9. Click the person to add them. You can add multiple people here if desired. Click Select to close that blade. 
10. Click Next, and then click Review+assign. Click Review+assign again.
11. Send the other person the path!

Granting Token Access to Storage Container

If you grant someone access to the storage container via a token, that person must use Microsoft Azure Storage Explorer to open that container. 

1. From the Azure web Portal, open Storage accounts, and then click the name of your storage account. 
2. Click Containers from the Data storage section on the menu panel. Then, from the main panel, click the name of the container you want to manage.
3. In the Settings section, click Shared access tokens. 
   1. Signing method: set to Account key. 
   2. You do not need to set the Signing key, and the Stored access policy is not used.
   3. Permissions: you can limit this token to only the specific operations you want to allow someone to perform. You must grant Read and List permissions as a minimum.
   4. Start and Expiry: you can set a start and end time of when someone is allowed to access this blob. If you are crossing time zones, be aware of those the difference.
   5. Allowed IP addresses: if the token user does have a dedicated IP address, then we recommend setting this option to increase security.
   6. Allowed protocols: - use HTTPS only.

4. Once you have set things as you need, click Generate SAS token and URL. The token and URL can be used to access the blob container. You can click the copy icon to the right of the token and URL to easily copy this information. 

Token Access to Specific Blob follows a similar process, but start by right-clicking specific blob, or click the ellipses and choose Generate SAS.

Review permissions periodically. You can add a recurring event to remind you to review access periodically.

Opening a storage container after you've been granted token access.

Azure Explorer must be used to access the the container via the access token. The person who needs to access the storage container must follow these steps:

1. Open Azure Storage Explorer.
2. Choose Attach to a resource.
3. Choose Blob container or directory.
4. Choose Shared access signature URL (SAS).
5. Paste in the Blob SAS URL that you received from the person who granted you access. That URL will not work in a browser.
6. Click OK.
7. The Blob container opens.

Programmatic access in Azure

Storage account administrators have storage account contributor and User Access Administrator roles and can therefore complete tasks programmatically in Azure. 

• Microsoft provides documentation and programmatic examples on a variety of programming languages: Azure Storage samples using Python - Azure Storage | Microsoft Learn.
• We specifically recommend that you review the Authentication options and the Blob examples.

Contents

Monitoring Usage

Azure Budgets

The UD Azure administrators create a Resource Group as the boundary for an account code setting, and you can use this boundary to set a budget for your archival storage. NOTE: A budget does not constrain your cost. If you exceed your budget, no services are stopped, no data is deleted, and costs will continue to accumulate. Setting a budget with multiple thresholds can alert you to incurred charges to your storage account.

Create an Azure budget

1. Create a action group:

   1. Log in to the Azure Web Browser.
   2. Click Storage accounts, and then click the name of your storage account.
   3. From the panel on the left, in the Monitoring section, click Alerts.
   4. Click the Create drop-down, and choose Action Group.
   5. Provide the action group name and display name, and click Next: Notifications.
   6. Choose Email/SMS message/Push/Voice, and type a name for your notification. 
   7. Click the pencil icon to specify your notification. Be sure to send the notification to an email address. Sending SMS messages incurs a charge. 
   8. Click OK, and then click Review + create. Click Create. 

2. Create the budget:

To create a budget, you need to fill in one or the other of “Action group” or “Alert Recipients”. We recommend alert groups as they can be reused. And if an address changes, you can update in one place instead of many places.

1. Log in to the Azure Web Browser.
2. Click Storage accounts, and then click the name of your storage account
3. Click the name of your Resource Group (listed in the Resource Group column). 
4. From the Cost management section, click Budgets. Then click Add.
5. Provide the details for your budget, then click Next.
6. Choose your alert condition's recipients, and click Create.

Choose Email/SMS message/Push/Voice, and be sure to send the notification to an email address. Sending SMS messages incurs a charge.

Monitor Costs

You can see how much money that you are spending in your Azure Blob Storage account by using the Cost analysis for your Resource Group. To do this:

1. Log in to the Azure Web Browser.
2. Click Storage accounts, and then click the name of your storage account
3. Click the name of your Resource Group (your resource group will most likely have an rg- preface). 
4. From the panel on the left, click Cost analysis.
5. A chart that indicates the costs incurred by your storage account appears.

Contents