Fifth Update of INDIGO-2
Last updated
Last updated
The Fifth Update of INDIGO-2 release contains:
New Features and Minor Changes
Patches and Bug Fixes
Upgrading an already deployed instance
Improved support for storage plugins
Documentation
support for data namespace browsing.
support for CDMI export attribute, it allows to expose to the client information about configured data access protocols, thanks to that end user is able to find the data-object with WebDav for example, but to be well understand: export attribute only informs about alternative protocols, it doesn't provide them, if by any chance the data object is available through WebDAV then export attribute will tell it
support for bunch of new QoS attributes including:
exposing information about data lifetime
exposing information about data retention policy
removed dependency on cdmi-s3-qos-ceph-provider module, the information which was earlier provided by
cdmi-s3-qos-ceph-provider now is partially read from configuration file (in case of static things) and partially is obtained directly from backed server through S3 protocol
Run with Docker
Packages
This is the first public release of the INDIGO CDMI StoRM plugin.
The CDMI StoRM plugin is currently released as a RHEL7 rpm:
This update provides
cloud-info-provider with GLUE 2.1 and Keystone V3 support
cloud-info-provider-indigo package containing JSON rendering templates and a script to interact with the CMDB using OpenID Connect
No manual intervention should be needed, but in case there are existing cron tasks they will have to take into account the following changes:
RHEL</br> yum clean all && yum update cloud-info-provider-indigo
Ubuntu</br>
apt-get update && apt-get install -V python-cloud-info-prov
RedHat
Ubuntu
ansible-role-cloud-info-provider: 0.1.0. Ansible role has been updated to work with the splitted packages.
Several bugfixes
Ansible role available in (always install latest version):
Documentation
The supported platforms
CentOS 7
Ubuntu 14.04
Ubuntu 16.04
CentOS 7
Ubuntu 16.04
Add Support OTC.
Support "DependsOn" TOSCA relationship.
Enable to specify a port range in endpoint ports.
Fix error loading an infrastructure from DB in case of invalid TOSCA.
Fix error getting IP info in OCCI conn.
Fix error in Azure conn creating a VM with only a public net attached.
Fix error in contextualization nodes with private leys.
More bugfixes and improvements...
The supported platforms
CentOS 7
Ubuntu 14.04
Ubuntu 16.04
CentOS 7
Ubuntu 14.04
Ubuntu 16.04
Docker Container:
Allow to set up custom ids for authN headers
The supported platforms
CentOS 6
CentOS 7
Ubuntu 14.04
Ubuntu 16.04
To know how install the im-java-api library please read the documentation available in:
To update the library you only need to substitute the im-java-api-X.X.X.jar
Tarballs:
This release provides fixes to some outstanding bugs and improvements:
New Features
The login button text can now be customized for local and SAML login
A privacy policy can now be linked to the IAM login page
Improved error pages rendering
SAML metadata can now be filtered according to certain conditions (e.g., SIRTFI compliance)
The organisation name is now included in the IAM dashboard top bar
IAM now implements a scope policy management API that allows to restrict the use of OAuth scopes only to selected users or group of users
Fixes
IAM now correctly enforces SAML metadata signature checks
The subject of IAM notification messages now includes the organization name
IAM Login Service can be deployed in two different ways:
as Docker container
as systemd daemon from precompiled packages
the IAM service is provided on the following DockerHub repositories:
indigoiam/iam-login-service
indigodatacloud/iam-login-service
The IAM service is executed by starting the docker container with the following command:
Since IAM 1.0.0, precompiled packages are available to install IAM Login services Supported platforms:
CentOS 7
Ubuntu 16.04
Installation
Installed the required Indigo IAM repository, install the IAM login service package.
On CentOS: </br>
On Ubuntu:
Run the service
The service is managed by Systemd, so to run it use:
Service Reference
A CLI client built on top of existing library to communicate with FutureGateway. It allows to submit & monitor tasks, upload & download files and query for available applications
A CLI client exposing basic FutureGateway functionality to the command line
Updated workflows and scripts to support ENES use case
Fixed a bug with incorrect handling of timezone offset when parsing timestamps
How to update: use provided Docker images versioned equally to the main libraries.
Sources
Ansible role indigo-dc/kepler v1.3 pushed to Ansiible Galaxy
Ansible role indigo-dc/kepler-batch v1.3 pushed to Ansible Galaxy
Docker image indigodatacloudapps/kepler:1.3 pushed to Docker Hub
Docker image indigodatacloudapps/kepler-batch:1.3 pushed to Docker Hub
CentOS7 source tarballs
Ubuntu16.04 source tarballs
Container
The new version includes:
the autogeneration of the UI for TOSCA based application
the access to runtime data in the UI.
Improved token management reducing the communication with IAM endpoints.
several bug fixes.
Provided details in the documentation. In short administrator should add the new modules from the Liferay control panel and disable/remove the previous.
CentOS 7
Ubuntu14.04
oidc-agent is a new system service ensuring that always a valid access token is available for
the command line.
This update brings many new features and bug fixes like
New Synergy service features:
added security support
New Synergy Scheduler Manager features:
Added improvement on queue management algorithm ( Synergy should scale up the oldest user requests from the queue)
Added a mechanism that performs user actions before Synergy deletes the VMs
Supported Operating Systems platforms:
CentOS 7
Ubuntu 16.04
Supported CMF (Cloud Management Framework) versions:
OpenStack v. Ocata
OpenStack v. Newton
OpenStack v. Mitaka (only CentOS 7)
OpenStack v. Liberty (only CentOS 7)
Packages:
CentOS7
Ubuntu 14.04
introducing SLA for storage resources
integration with ONEDATA to obtain user space
enable CMDB-based recognition of service provider
improved performance and UX
fixing reported bugs
Container:
Several bugfixes
Ansible role available in (always install latest version):
Documentation
The supported platforms
CentOS 7
Ubuntu 14.04
Ubuntu 16.04
CentOS 7
Ubuntu 16.04
oidc-agent is a new system service ensuring that always a valid access token is available for
the command line.
Highlights of this update are:
interface to the newly release oidc-agent
CentOS 7
Ubuntu 16.04
Container
The current update adds support for Azure, increased support for Openstack and better configuration experience
Docker Container:
The updated version of TOSCA-parser addresses some minor bugfixes
In order to update the packages please use:
For CentOS 7:
yum clean all && yum update tosca-parser
For Ubuntu 14:04/16.04:
apt-get update && apt-get install python-tosca-parser
CentOS7
Ubuntu 14.04
Ubuntu 16.04
The updated version of TOSCA types contains various improvements and adds some use cases
Add configuration parameters for Chronos and Marathon nodes
Update template for Long-Running Services
Improve Galaxy configuration.
Fixing properties types for Spark
Major improvements for Dariah use-case
Update tosca types for cms use-case
Added type for spark on mesos.
Update Mesos types
Updated Marathon app definition
CentOS7
Ubuntu 14.04
Ubuntu 16.04
udocker 1.1.0 includes additional container execution engines based on fakechroot and runC
which provide alternative methods to execute the containers based on different approaches.
These new engines can also deliver additional performance for applications that make intensive
use of system calls.
Bug fixes and new features:
Support image names prefixed by registry similarly to docker
Add execution engine selection logic
Add fr execution engine based on shared library interception
Add rc execution engine based on rootless namespaces
Improve proot tmp files cleanup on non ext filesystems
Improve search returning on Docker repositories
Improve runC execution portability
Add environment variable UDOCKER_KEYSTORE
Prevent creation of .udocker when UDOCKER_KEYSTORE is used
Information for installation and upgrade is available in the GitBook documentation at
Binary Tarball
CentOS 7
Ubuntu 14.04
Ubuntu 16.04
This update is just a bug fix release:
if caching is uses (which is done by default) access tokens are invalidated way to fast, this is now fixed
a workaroung is to disable caching by setting: oidc.cache_duration = none (version 1.1.0 to 1.2.0)
CentOS 7
Ubuntu 14.04
Ubuntu 16.04
This update provides integration with oidc-agent, so it can be used instead of storing an access token in an environmental varible
CentOS 7
Ubuntu 14.04
Ubuntu 16.04
Docker container.
Authentication for Onedata through IAM service and automatic process for refreshing token.
Some connection errors broke up the process and were catched and fixed.
A null lenght of oneprovider metrics broke up the process and were validated and fixed.
Authentication for Onedata can be done with a valid IAM token. This token and a refresh token is requested by a script (Client application credentials are required) and passed to the monitoring agent as input parameters.
For Centos 6/7 and Ubuntu 14/16 download the respective installer and execute it. For other Linux distributions, download and copy into desired directory where monitoring agent is installed.
Centos:
Ubuntu:
CentOS7
Ubuntu14.04
Ubuntu16.04
Add token introspection and Caching:
Add files to enable ansible deployment:
GET Test Case Refactor:
POST Test Case Refactor:
Add branch coverage to coveralls check:
Upgrade base Docker image to CentOS7:
Update documentation in docker/README.md:
Update Partition range for new deployments:
Improve documentation following second round of 'Early Adopter' testing:
- CentOS7 based image
This is the first release of the Analytics Portlets - for more details see the
SLF4J: Class path contains multiple SLF4J bindings. ->
Upgrade Guide:
Installation guide available at -
Ansible role is available here:
For more details please read
Documentation is available at:
- cloud-info-provider: Add GLUE 2.1 support
- cloud-info-provider: Add Keystone v3 auth support Splitting repositories/packages
- cloud-info-provider-indigo: Integrate CMDB calls with IAM-based authentication
- All INDIGO-specific content (JSON templates, send-to-cmdb script and documentation). Dependent on cloud-info-provider.
- Upstream version of the cloud-info-provider including changes made in the context of INDIGO.
Assure not to delete non existing resources ->
Show statusReason in clues log ->
Bugfix deleting nodes ->
Fix error in mesos plugin in case no data returned by _obtain_mesos_jobs ->
Improve getting Public network name in OCCI: .
Improve Ansible contextualization to scale more than 50 VMs: .
Add Support Custom instance types in Google Conn: .
Implements Delete SGs in ONE conn only in last VM: .
Fix error getting contmsg output in case of use colors: .
Fix error in SSH module in sftp_put_dir if src dir ends with /: .
Fix SSL verification error in windows: .
Enable to add IPs in OCCI conn in AlterVM: .
Fix error in OpenStack conn trying to assing a floating IP to VM with public IP: .
Wait SSH access to VMs only in one ctxt task: .
Create only one storage account per Infrastructure instead of per VM in Azure conn: .
Improve save data in VM creation process: .
Fix Error creating NGS in Azure conn: .
Return false if VM does not exists in updateVMInfo: .
Update Ansible roles in Master node in reconfiguration: .
Fix error in Azure conn creating subnet: .
Fix error in Context. process in basic test in SSH tests: .
Add compatiblity with Ansible 2.4.0: .
Fix error in ctxt process in some cases in OpenStack sites: .
Enable to specify private network_name without .PRIVATE at the end: .
Fix Error using funtions to set a value of a BlockStorage: .
Enable to specify not using a private ip in a Cumpute node: .
Allow to set up custom ids for authN headers
Fix wrong auth_version value for Openstack V3 (with password) authorization header
Support subscription_id param in Azure auth header
More details:
More information about bug fixes and other developments can be found on
Also a Puppet module is provided to simplify the installation and setup. This module leveraging on the precompiled packages.
See the gitbook admin guide for all configuration variables description.
Packages and repo files are hosted on public repository.
IAM gitbook:
Please read the
Stable, certified rpm/deb releases suitable for use in production can be found at:
Docker image can be found at: tag v1.1.0-latest
- Support Runtime Data of the tasks
- Ajax call should be async
- Default paramter file generation
- Javascript dependency error
- Remove or improve alert messages
- Caching keys for token validation
- Default values in TOSCA template should be selected in Customisable Application Portlet
- Float field in the UI should accept integer
- Admin portlet should list task for all users
- UserInfo endpoint called with expired token
- OneProvider dns only name
More information can be found in the "Upgrade to a new release" section of the
for more information please see:
Complete list of issues is available at:
Service Reference Card:
Update/Upgrade Synergy packages:
add onedata space metric to storage SLA -
fix storage class select -
upgrade engine 1.3 -- authority redesing -
adding ONEDATA URL option to docker
fixing type of resource for computing SLA
adding Storage SLA
adding recognision of CMDB authorities
OneData integration: fetching user spaces
For upgrade please read the section "Upgrade procedure" in
Assure not to delete non existing resources ->
Show statusReason in clues log ->
Bugfix deleting nodes ->
Fix error in mesos plugin in case no data returned by _obtain_mesos_jobs ->
for more information please see:
- add oidc-agent support
Documentation is avalable at - chapter "Setting the Access Token"
Enable the users to retrieve the underlining IaaS IDs ()
Evaluation of the TOSCA Concat function ()
Retrieval of TOSCA attributes for Marathon Deployments ()
Deploy on Azure ()
Support for Openstack regions different from the default ones ()
Add support for local volumes for Mesos tasks ()
Please read the "Upgrade guide":
Fix error getting relationshps in case of custom_def capability:
Fix error getting a boolean capability property with value false:
Please see the
Just update/install also documented here -
Just update/install install also documented here -