Class: AzureBlob::Client

Inherits:
Object
  • Object
show all
Defined in:
lib/azure_blob/client.rb

Overview

AzureBlob Client class. You interact with the Azure Blob api through an instance of this class.

Instance Method Summary collapse

Constructor Details

#initialize(account_name:, access_key: nil, principal_id: nil, container:, host: nil, **options) ⇒ Client

Returns a new instance of Client.



19
20
21
22
23
24
25
26
27
28
 
# File 'lib/azure_blob/client.rb', line 19

def initialize(account_name:, access_key: nil, principal_id: nil, container:, host: nil, **options)
  @account_name = 
  @container = container
  @host = host
  @cloud_regions = options[:cloud_regions]&.to_sym || :global
  @access_key = access_key
  @principal_id = principal_id
  @use_managed_identities = options[:use_managed_identities]
  signer unless options[:lazy]
end

Instance Method Details

#append_blob_block(key, content, options = {}) ⇒ Object

Append a block to an Append Blob

Calls to Append Block

Options:

:content_md5

Will ensure integrity of the upload. The checksum must be a base64 digest. Can be produced with OpenSSL::Digest::MD5.base64digest. The checksum must be the checksum of the block not the blob.



307
308
309
310
311
312
313
314
315
316
317
318
 
# File 'lib/azure_blob/client.rb', line 307

def append_blob_block(key, content, options = {})
  uri = generate_uri("#{container}/#{key}")
  uri.query = URI.encode_www_form(comp: "appendblock")

  headers = {
    "Content-Length": content_size(content),
    "Content-Type": options[:content_type],
    "Content-MD5": options[:content_md5],
  }.merge(additional_headers(options))

  Http.new(uri, headers, signer:).put(content)
end

#blob_exist?(key, options = {}) ⇒ Boolean

Returns a boolean indicating if the blob exists.

Calls to Get Blob Properties

Returns:

  • (Boolean) 


184
185
186
187
188
 
# File 'lib/azure_blob/client.rb', line 184

def blob_exist?(key, options = {})
  get_blob_properties(key, options).present?
rescue AzureBlob::Http::FileNotFoundError
  false
end

#commit_blob_blocks(key, block_ids, options = {}) ⇒ Object

Commits the list of blocks to a blob.

Calls to Put Block List

Takes a key (path) and an array of block ids

Options:

:content_md5

This is the checksum for the whole blob. The checksum is saved on the blob, but it is not validated! Add a checksum for each block if you want Azure to validate integrity.



357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
 
# File 'lib/azure_blob/client.rb', line 357

def commit_blob_blocks(key, block_ids, options = {})
  block_list = BlockList.new(block_ids)
  content = block_list.to_s
  uri = generate_uri("#{container}/#{key}")
  uri.query = URI.encode_www_form(comp: "blocklist")

  headers = {
    "Content-Length": content_size(content),
    "Content-Type": options[:content_type],
    "x-ms-blob-content-md5": options[:content_md5],
    "x-ms-blob-content-disposition": options[:content_disposition],
  }.merge(additional_headers(options))

  Http.new(uri, headers, signer:, **options.slice(:metadata, :tags)).put(content)
end

#container_exist?(options = {}) ⇒ Boolean

Returns a boolean indicating if the container exists.

Calls to Get Container Properties

Returns:

  • (Boolean) 


221
222
223
 
# File 'lib/azure_blob/client.rb', line 221

def container_exist?(options = {})
  get_container_properties(options = {}).present?
end

#copy_blob(key, source_key, options = {}) ⇒ Object

Copy a blob between containers or within the same container

Calls to Copy Blob From URL

Parameters:

  • key: destination blob path

  • source_key: source blob path

  • options: additional options

    • source_client: AzureBlob::Client instance for the source container (optional) If not provided, copies from within the same container



92
93
94
95
96
97
98
99
100
101
102
103
104
 
# File 'lib/azure_blob/client.rb', line 92

def copy_blob(key, source_key, options = {})
  source_client = options.delete(:source_client) || self
  uri = generate_uri("#{container}/#{key}")

  source_uri = source_client.signed_uri(source_key, permissions: "r", expiry: Time.at(Time.now.to_i + 300).utc.iso8601)

  headers = {
    "x-ms-copy-source": source_uri.to_s,
    "x-ms-requires-sync": "true",
  }.merge(additional_headers(options))

  Http.new(uri, headers, signer:, **options.slice(:metadata, :tags)).put
end

#create_append_blob(key, options = {}) ⇒ Object

Creates a Blob of type append.

Calls to Put Blob

You are expected to append blocks to the blob with append_blob_block after creating the blob. Options:

:content_type

Will be saved on the blob in Azure.

:content_disposition

Will be saved on the blob in Azure.



284
285
286
287
288
289
290
291
292
293
294
295
296
 
# File 'lib/azure_blob/client.rb', line 284

def create_append_blob(key, options = {})
  uri = generate_uri("#{container}/#{key}")

  headers = {
    "x-ms-blob-type": "AppendBlob",
    "Content-Length": 0,
    "Content-Type": options[:content_type],
    "Content-MD5": options[:content_md5],
    "x-ms-blob-content-disposition": options[:content_disposition],
  }.merge(additional_headers(options))

  Http.new(uri, headers, signer:, **options.slice(:metadata, :tags)).put(nil)
end

#create_block_blob(key, content, options = {}) ⇒ Object

Create a blob of type block. Will automatically split the the blob in multiple block and send the blob in pieces (blocks) if the blob is too big.

When the blob is small enough this method will send the blob through Put Blob

If the blob is too big, the blob is split in blocks sent through a series of Put Block requests followed by a Put Block List to commit the block list.

Takes a key (path), the content (String or IO object), and options.

Options:

:content_type

Will be saved on the blob in Azure.

:content_disposition

Will be saved on the blob in Azure.

:content_md5

Will ensure integrity of the upload. The checksum must be a base64 digest. Can be produced with OpenSSL::Digest::MD5.base64digest. The checksum is only checked on a single upload! To verify checksum when uploading multiple blocks, call directly put_blob_block with a checksum for each block, then commit the blocks with commit_blob_blocks.

:block_size

Block size in bytes, can be used to force the method to split the upload in smaller chunk. Defaults to AzureBlob::DEFAULT_BLOCK_SIZE and cannot be bigger than AzureBlob::MAX_UPLOAD_SIZE



51
52
53
54
55
56
57
 
# File 'lib/azure_blob/client.rb', line 51

def create_block_blob(key, content, options = {})
  if content_size(content) > (options[:block_size] || DEFAULT_BLOCK_SIZE)
    put_blob_multiple(key, content, **options)
  else
    put_blob_single(key, content, **options)
  end
end

#create_container(options = {}) ⇒ Object

Create the container

Calls to Create Container



228
229
230
231
232
233
234
235
236
237
 
# File 'lib/azure_blob/client.rb', line 228

def create_container(options = {})
  uri = generate_uri(container)
  headers = {}
  headers[:"x-ms-blob-public-access"] = "blob" if options[:public_access]
  headers[:"x-ms-blob-public-access"] = options[:public_access] if [ "container", "blob" ].include?(options[:public_access])
  headers.merge!(additional_headers(options))

  uri.query = URI.encode_www_form(restype: "container")
  response = Http.new(uri, headers, signer:).put
end

#delete_blob(key, options = {}) ⇒ Object

Delete a blob

Calls to Delete Blob

Takes a key (path) and options.

Options:

:delete_snapshots

Sets the value of the x-ms-delete-snapshots header. Default to include



115
116
117
118
119
120
121
122
123
 
# File 'lib/azure_blob/client.rb', line 115

def delete_blob(key, options = {})
  uri = generate_uri("#{container}/#{key}")

  headers = {
    "x-ms-delete-snapshots": options[:delete_snapshots] || "include",
  }.merge(additional_headers(options))

  Http.new(uri, headers, signer:).delete
end

#delete_container(options = {}) ⇒ Object

Delete the container

Calls to Delete Container



242
243
244
245
246
 
# File 'lib/azure_blob/client.rb', line 242

def delete_container(options = {})
  uri = generate_uri(container)
  uri.query = URI.encode_www_form(restype: "container")
  response = Http.new(uri, additional_headers(options), signer:).delete
end

#delete_prefix(prefix, options = {}) ⇒ Object

Delete all blobs prefixed by the given prefix.

Calls to List blobs followed to a series of calls to Delete Blob

Takes a prefix and options

Look delete_blob for the list of options.



133
134
135
136
 
# File 'lib/azure_blob/client.rb', line 133

def delete_prefix(prefix, options = {})
  results = list_blobs(prefix:)
  results.each { |key| delete_blob(key) }
end

#generate_uri(path) ⇒ Object

Return a URI object to a resource in the container. Takes a path.

Example: generate_uri(“#{container}/#{key}”)



251
252
253
254
255
256
257
258
 
# File 'lib/azure_blob/client.rb', line 251

def generate_uri(path)
  # https://github.com/Azure/azure-storage-ruby/blob/master/common/lib/azure/storage/common/service/storage_service.rb#L191-L201
  encoded_path = CGI.escape(path.encode("UTF-8"))
  encoded_path = encoded_path.gsub(/%2F/, "/")
  encoded_path = encoded_path.gsub(/%5C/, "/")
  encoded_path = encoded_path.gsub(/\+/, "%20")
  URI.parse(File.join(host, encoded_path))
end

#get_blob(key, options = {}) ⇒ Object

Returns the full or partial content of the blob

Calls to the Get Blob endpoint.

Takes a key (path) and options.

Options:

:start

Starting point in bytes

:end

Ending point in bytes



71
72
73
74
75
76
77
78
79
 
# File 'lib/azure_blob/client.rb', line 71

def get_blob(key, options = {})
  uri = generate_uri("#{container}/#{key}")

  headers = {
    "x-ms-range": options[:start] && "bytes=#{options[:start]}-#{options[:end]}",
  }.merge(additional_headers(options))

  Http.new(uri, headers, signer:).get
end

#get_blob_properties(key, options = {}) ⇒ Object

Returns a Blob object without the content.

Calls to Get Blob Properties

This can be used to obtain metadata such as content type, disposition, checksum or Azure custom metadata. To check for blob presence, look for ‘blob_exist?` as `get_blob_properties` raises on missing blob.



173
174
175
176
177
178
179
 
# File 'lib/azure_blob/client.rb', line 173

def get_blob_properties(key, options = {})
  uri = generate_uri("#{container}/#{key}")

  response = Http.new(uri, additional_headers(options), signer:).head

  Blob.new(response)
end

#get_blob_tags(key, options = {}) ⇒ Object

Returns the tags associated with a blob

Calls to the Get Blob Tags endpoint.

Takes a key (path) of the blob.

Returns a hash of the blob’s tags.



197
198
199
200
201
202
203
 
# File 'lib/azure_blob/client.rb', line 197

def get_blob_tags(key, options = {})
  uri = generate_uri("#{container}/#{key}")
  uri.query = URI.encode_www_form(comp: "tags")
  response = Http.new(uri, additional_headers(options), signer:).get

  Tags.from_response(response).to_h
end

#get_container_properties(options = {}) ⇒ Object

Returns a Container object.

Calls to Get Container Properties

This can be used to see if the container exist or obtain metadata.



210
211
212
213
214
215
216
 
# File 'lib/azure_blob/client.rb', line 210

def get_container_properties(options = {})
  uri = generate_uri(container)
  uri.query = URI.encode_www_form(restype: "container")
  response = Http.new(uri, additional_headers(options), signer:, raise_on_error: false).head

  Container.new(response)
end

#list_blobs(options = {}) ⇒ Object

Returns a BlobList containing a list of keys (paths)

Calls to List blobs

Options:

:prefix

Prefix of the blobs to be listed. Defaults to listing everything in the container.

:max_results

Maximum number of results to return per page.



147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
 
# File 'lib/azure_blob/client.rb', line 147

def list_blobs(options = {})
  uri = generate_uri(container)
  query = {
    comp: "list",
    restype: "container",
    prefix: options[:prefix].to_s.gsub(/\\/, "/"),
  }
  query[:maxresults] = options[:max_results] if options[:max_results]
  uri.query = URI.encode_www_form(**query)

  fetcher = ->(marker) do
    query[:marker] = marker
    query.reject! { |key, value| value.to_s.empty? }
    uri.query = URI.encode_www_form(**query)
    response = Http.new(uri, additional_headers(options), signer:).get
  end

  BlobList.new(fetcher)
end

#put_blob_block(key, index, content, options = {}) ⇒ Object

Uploads a block to a blob.

Calls to Put Block

Returns the id of the block. Required to commit the list of blocks to a blob.

Options:

:content_md5

Must be the checksum for the block not the blob. The checksum must be a base64 digest. Can be produced with OpenSSL::Digest::MD5.base64digest.



330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
 
# File 'lib/azure_blob/client.rb', line 330

def put_blob_block(key, index, content, options = {})
  block_id = generate_block_id(index)
  uri = generate_uri("#{container}/#{key}")
  uri.query = URI.encode_www_form(comp: "block", blockid: block_id)

  headers = {
    "Content-Length": content_size(content),
    "Content-Type": options[:content_type],
    "Content-MD5": options[:content_md5],
  }.merge(additional_headers(options))

  Http.new(uri, headers, signer:).put(content)

  block_id
end

#signed_uri(key, permissions:, expiry:, **options) ⇒ Object

Returns an SAS signed URI

Takes a

  • key (path)

  • A permission string (+“r”+, “rw”)

  • expiry as a UTC iso8601 time string

  • options



267
268
269
270
271
 
# File 'lib/azure_blob/client.rb', line 267

def signed_uri(key, permissions:, expiry:, **options)
  uri = generate_uri("#{container}/#{key}")
  uri.query = signer.sas_token(uri, permissions:, expiry:, **options)
  uri
end