NAME

Data::Resolver - resolve keys to data

VERSION

This document describes Data::Resolver version {{[ version ]}}.

SYNOPSIS

# generate is a single entry point useful to instantiate from
# metadata/configurations
use Data::Resolver 'generate';

my $spec = { -factory => 'resolver_from_tar',
             path => '/to/archive.tar' };
my $dr_tar = generate($spec);

my $dr_dir = generate({ -factory => 'resolver_from_dir',
                        path => '.' });

# functions can be imported and used directly though
use Data::Resolver qw< resolver_from_tar resolver_from_dir >;
my $dr_tar = resolver_from_tar(path => 'to/somewhere.tar');

# getting stuff is easy, this is how to get the default
# representation
my ($thing, $meta) = $dr_something->($key);
my $type = $meta->{type};
if    ($type eq 'file')       { say 'got a file'       }
elsif ($type eq 'filehandle') { say 'got a filehandle' }
elsif ($type eq 'data')     ) { say 'got data'         }
elsif ($type eq 'error') { ... }

# you can specify the type and just get the value
my $data = $dr_something->($key, 'data');
my $path = $dr_something->($key, 'file');
my $fh   = $dr_something->($key, 'filehandle');

# it's possible to get a list of available keys
my $list_aref = $dr_something->(undef, 'list');

DESCRIPTION

While coding, two problems often arise:

• Using several modules, there can be a variety of ways on how they get access to data. Many times they support reading from a file, but often times they expect to receive data (e.g. JSON::PP). Other times modules an be OK with both, and even accept filehandles.
• Deciding on where to store data and what to use as a source can be limiting, especially when multiple things might be needed. What is best at that point? A directory? An archive? A few URLs?

This module aims at providing a way forward to cope for both problems, by providing a unified interface that can get three types of data types (i.e. data, file, or filehandle) while at the same time providing a very basic interface that can be backed by several different fetching approaches, like reading from a directory, taking items from an archive, or download stuff on the fly from a URL.

The Resolver Interface

A valid resolver is a function that supports at least the following interface:

• The resolver has the following signature:

    my $resolver = sub ($key, [$type]) { ... }
  

  where $key is a mandatory parameter providing the key that we want to resolve to a data representation, and $type is an optional parameter that specifies what representation is needed.

• When called in list context, two items are provided back, i.e. the value and the metadata:

    my ($value, $meta) = $resolver->(@args);
  

  The $meta is a hash reference that contains at least one key type, indicating the type of the $value. Allowed types are at least the following:

  • data

    The $value is directly data. It might be provided either as a plain scalar, or as a reference to a plain scalar (ref() will help disambiguate the two cases).

  • error

    The $value should be ignored because an error occurred during retrieval. When the resolver is set for throwing an exception, this is never returned.

  • file

    The $value represents a file path in the filesystem.

  • filehandle

    The $value is a filehandle, suitable for reading. Characteristics of thi filehandle may vary, although it SHOULD support seeking.

• When called in scalar context, only the $value is provided back:

    my $value = $resolver->(@args);
  

  In this case it is usually better to also provide a type as the second argument, unless the default return type for the resolver is already known in advance.

• The following invocation provides back a list of all supported keys:

    my $list = $resolver->(undef, 'list');
  

Examples:

# get list of supported keys, as an array ref
my $list = $resolver->(undef, 'list');

# get value associated to key 'foo', as raw data
my $data = $resolver->(foo => 'data');

# get value and metadata, decide later how to use them
my ($value, $meta) = $resolver->('foo');
if ($meta->{type} eq 'file') { ... }

Stock Factories

The module comes with a few stock factory functions to generate resolvers in a few cases:

• A directory in the filesystem, via "resolver_from_dir".
• A TAR archive, via "resolver_from_tar".
• A list of resolvers, via "resolver_from_alternatives". This allows e.g. looking for a resolution of the key from multiple sources (possibly of different kinds).

INTERFACE

The interface provided by this module can be broadly divided into three areas:

• factories to generate resolvers;
• transformers to ease turning a data representation into another (e.g. turning data into a file or a filehandle)
• utilities for coding new resolvers/resolver factories.

Factories

These functions generate resolvers.

generate

my $resolver = generate($specification_hash);

Generate a new resolver based on a hash containing a specification. The following meta-keys are supported:

• -factory

  The name of the factory function (e.g. "resolver_from_tar").

• -package

  The package where the factory function above is located. By default this is Data::Resolver.

• --recursed-args

  A sub-hash where values are array references, holding sub-specifications that are generated recursively via generate itself. The key and the resulting array are then inserted as new keys/value pairs in the hash.

All the rest of the hash is passed to the factory function as key/value pairs.

Example:

my $spec = { -factory => 'resolver_from_tar',
             path => '/to/archive.tar' };
my $dr_tar = generate($spec);

my $spec = {
   -factory => 'resolver_from_alternatives',
   -recursed => {
      alternatives => [
         { -factory => resolver_from_tar => archive => $tar },
         { -factory => resolver_from_dir => root    => $dir },
      ],
   }
};
my $dr_multi = generate($spec);

resolver_from_alternatives

my $dr = resolver_from_alternatives(%args);

Generate a resolver that wraps other resolvers and tries them in sequence, until the first supporing the input key.

It cares about two keys in %args:

• alternatives

  Accepts a reference to an array of sub-resolvers (i.e. CODE references) or sub-resolver specifications (which will be instantiated via "generate").

• throw

  If set to a true value, raise an exception in case of errors.

The list type is supported for the undef key only. It replicates the call over all alternatives, aggregating the result in the order they appear while filtering out duplicates (it does not try to normalize keys in any way, so this might give practical duplicates out).

The search for a key is performed in the same order as the sub-resolvers appear in alternatives; when a result is found, it is returned. Exceptions from sub-resolvers are trapped.

If throw is set, errors will raise exceptions thrown as hashes, via "resolved_error"/"resolved". This happens in two cases:

• the call for type list does not provide an undef key. In this case, the error code is set to 400.
• the call for any other type does not provide a result back. In this case, the error code is set to 404.

resolver_from_dir

my $dr = resolver_from_dir(%args);

Generate a resolver that serves files from a directory in the local filesystem. It supports the following keys in %args:

• path

• root

  The path to the directory containing the files. root takes precedence over path.

• throw

  If set to a true value, raise an exception in case of errors.

Errors are handled via "resolved_error"/"resolved", including raising exceptions.

The call to type list with an undef key will generate a lit of all files in the subtree starting from path/root. As an extension, it's also possible to pass the name of a sub-directory and get only that subtree back; this is prone to errors if the sub-directory does not exist (error code 404) or is a file instead of a directory (error code 400).

The call for other types will resolve the key and provide back the requested type, defaulting to type file for effort minimization. The code tries to restrict looking for the file only inside the sub-tree but you should check by yourself if this is really critical (patches welcome!).

If the key cannot be found, error code 404 is set; if the key refers to a directory, error code 400 is set; if the type cannot be handled via "transform", error code 400 is set.

resolver_from_tar

my $dr = resolver_from_tar(%args);

Generate a resolver that serves file from a TAR file in the local filesystem. It supports the following keys in %args:

• archive

• path

  The path to the TAR file. archive takes precedence over path.

• throw

  If set to a true value, raise an exception in case of errors.

Errors are handled via "resolved_error"/"resolved", including raising exceptions.

The call for type list only supports key undef and will lead to error code 400 otherwise.

The call for any other type will look for the key and return the requested type, defaulting to type data. Two keys are actually searched, to cater for the equivalence of path/to/file and ./path/to/file; this means that it's possible to ask for somefile and get back the contents of ./somefile, or vice-versa.

If a file for a key cannot be found, error code 404 is returned. This also applies if the key is present, but represents a directory item.

Type transformation is performed via "transform"; unsupported types will lead to error code 400 after the search and extraction of data from the archive (i.e. there is no attempt to pre-validate the type and this is by design).

resolver_from_passthrough

my $dr = resolver_from_passthrough(%args);

Generate a minimal, fake-like resolver that always returns what is provided. Arguments %args are added to the return value via "resolved", so key throw might lead to exceptions.

In the generated resolver, the type defaults to undef. No attempt at validation is done, by design.

Transformers

data_to_fh

my $data = 'Some thing';
my $fh1 = data_to_fh($data);  # first way
my $fh2 = data_to_fh(\$data); # second way

Gets a scalar or a scalar reference and provides a filehandle back, suitable for reading/seeking.

data_to_file

my $data = 'Some thing';
my $path1 = data_to_file($data);  # plain scalar in
my $path2 = data_to_file(\$data); # scalar reference in
my $persistent_path = data_to_file($data, 1);

Gets a scalar or a scalar reference and saves it to a temporary file in the filesystem. A second parameter, when true, makes it possible to persist the file after the process exists (the file would be removed otherwise).

fh_to_data

open my $fh, '<:raw', $some_path or die '...';
my $data = fh_to_data($fh);

Slurps data from a filehandle. The filehandle is not changed otherwise, so it's up to the caller to set the right Perl IO layers if needed.

fh_to_file

open my $fh, '<:raw', $some_path or die '...';
my $path = fh_to_file($fh);
my $persistent_path = fh_to_file($fh, 1);

Slurps data from a filehandle and saves it into a temporary file (or persistent, if the second parameter is present and true).

file_to_data

my $data = file_to_data($path);

Slurps data from a file in the filesystem. Data are read in raw mode.

file_to_fh

my $fh = file_to_fh($path);

Opens a file in the filesystem in read raw mode.

transform

my $ref_to_that = transform($this, $this_type, $that_type, $keep = 0);

Treat input $this as having $this_type and return it as a reference to a $that_type.

Input and output types can be:

• fh

• filehandle

  the input is a file handle

• data

  the input is raw data

• file

• path

  the input is a path in the filesystem.

NOTE the return value is a reference to the target data form, to avoid transferring too much data around.

The interface also supports an additional parameter $keep, defaulting to false. This asks to generate a file that will be persisted when the process exits, when the target type is a file (by default, the file would be deleted).

Utilities For New Resolvers

This module comes with two non-trivial resolvers, one for wrapping a directory and another one for tar archives. There can be other possible resolvers, e.g. using different archive formats (like ZIP), leveraging any file format that supports carrying metadata (like PDF, or many image formats), or wrapping remote resources (plain HTTP or some fancy API).

These functions help complying with the output API of a resolver, i.e.:

• throw an exception when errors occur and the resolver was created with throw parameter set;
• return just the content in scalar context;
• return the content and additional metadata in list context.

A typical way of using these function is like this:

sub resolver_for_whatever (%args) {
   my $OK = resolved_factory($args{throw});
   my $KO = resolved_error_factory($args{throw});
   return sub ($key, $type = 'xxx') {
      return $KO->(400 => 'Wrong inputs!') if $some_error;
      return $OK->($data, type => 'data');
   };
}

resolved

return resolved($throw, $value, $meta_as_href);
return resolved($throw, $value, %meta);

Throw an exception if $throw is true and metadata have type set to error.

Otherwise, return $value if called in scalar context.

Otherwise, return a list with $value and a hash reference with the metadata.

resolved_error

return resolved_error($throw, $code, $message, $meta);
return resolved_error($throw, $code, $message, %meta);

If an error has to be returned, this is a shorthand to integrate the optional metadata with a code and a message. If $throw is set, an exception is thrown.

resolved_error_factory

my $error_return = resolved_error_factory($throw);

Wrap "resolved_error" with the specific value for $throw. This can be useful because whether a resolver should throw exceptions or not is usually set at resolver creation time, so it makes sense to wrap this characteristic.

resolved_factory

my $return = resolved_factory($throw);

Wrap "resolved" with the specific value for $throw. This can be useful because whether a resolver should throw exceptions or not is usually set at resolver creation time, so it makes sense to wrap this characteristic.

BUGS AND LIMITATIONS

Minimum perl version 5.24 because reasons (it's been around since 2016 and signatures just make sense).

Report bugs through Codeberg (patches welcome) at https://codeberg.org/polettix/Data-Resolver/issues.

AUTHOR

Flavio Poletti flavio@polettix.it

COPYRIGHT AND LICENSE

Copyright 2023 by Flavio Poletti flavio@polettix.it

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.