Sixth Update of INDIGO-1
Last updated
Last updated
The Sixth Update of INDIGO-1 release contains:
This release provides new functionality and some bug fixes like the possible for users to link external authentication accounts
(Google, SAML) to the user IAM account and to register at the IAM starting from an external authentication
Features:
- As an authenticated user, I can link an external SAML or OIDC account to my IAM account
- As an autenticated user, I can change my password
- It is now possible to register at the IAM using one of supported external authentication mechanism
- The IAM now exposes an authority management endpoint (integrated in the dashboard) that allows to assign/remove administrative rights to/from users
- The IAM now provides a refactored SAML WAYF service that remembers the identity provider chosen by the user.
Bug fixes:
- The IAM now provides a refactored SAML WAYF service that remembers the identity provider chosen by the user.
- Group creation in the dashboard now behaves as expected
- Editing first name/family name fails on the dashboard
, -Improved token exchange documentation
- SCIM: the SCIM create group endpoint should not require clients to provide a uuid
- The IAM discovery endpoint does not advertise support for the resource owner password credential flow
- It is now possible to set custom SAML maxAssertionTime and maxAuthenticationAge to customize how the SAML filter should check incoming SAML responses and assertions
More information about bug fixes and other developments can be found on our
docker pull indigodatacloud/iam-login-service
docker stop iam-login-service
docker rm iam-login-service
docker run \
--name iam-login-service --net=iam -p 8080:8080 \
--env-file=/path/to//iam-login-service/env \
-v /path/to//keystore.jks:/keystore.jks:ro \
indigodatacloud/iam-login-service:indigo_1
Docker Container:
This update fixes an important bug which prevent the correct validation of the token in some conditions.
In order to use the latest version, 1.2.1, it is requested:
To upgrade a running service people have to remove previous jar files and deploy upload the new packages.
No configuration is requested, it is preserved from the previous installation.
Note: to remove the old version and copy the new files it is possible to use the Ansible Playbookavailable at:
CentOS 7
Ubuntu14.04
Onedata 3.0.0.11 INDIGO - DataCloud release improved performance and stability of all Onedata services, as compared to the initial release. New features, such as public shares and file/directory/space metadata support were added. Furthermore, data stored in Onezone and Oneprovider is now automatically processed when upgrading to new version.
A complete changelog reflecting all bug fixed and new features is included in every Onedata repository on github in CHANGELOG.md. Furthermore, most notable issues were features were selected and included in this document.
Onezone
VFS-2633 Use specific package builders
VFS-2582 Using GUI fix for blank notifications
VFS-2390 Upgrade rebar to version 3
Update one panel for extended configuration options
Add ONEPANEL_DEBUG_MODE env variable to release docker entrypoint
Oneclient
VFS-2400 Update to new ceph and aws libraries
Fix storage detection in case of unsupported space
VFS-1963 Improve automatic storage discovery
VFS-2316 Integrate new etls version.
VFS-2250 Add base62 encoding and decoding to tokenHandler
VFS-2272 Give precendence to env AUTHORIZATION_TOKEN.
VFS-2270 Print out the hostname of client's provider.
VFS-2215 Remove the file immediately on unlink.
Oneprovider
VFS-2617 Changed metadata submenu to metadata panel
VFS-2180 Improve links conflict resolution
VFS-2180 Improve dbsync implementation
VFS-2180 Use gen_server2 instead of erlang's gen_server module
VFS-2390 Fix handlers specification in REST API
VFS-2390 Update rebar to version 3
VFS-2180 Allow for concurrent file creation
Update interfaces
Increase system performance
Update one panel for extended configuration options
Add ONEPANEL_DEBUG_MODE env variable to release docker entrypoint
VFS-2395 Added scrolling to element when clicking settings icon.
Turn off HSTS by default, allow configuration via app.config
Update file consistency management
Enable metadata caching
Extend support for user-defined metadata
Update Web_GUI and REST API
Enable Symmetric Multiprocessing
VFS-2773 Listen to more changes in /changes api and add a few new tests.
VFS-2764 Fix directories having 0B size in GUI
VFS-2764 Fix size of files being zero right after upload
VFS-2662 Append new files to the beginning of the files list
VFS-2662 Implement file creation compatible with pagination model
VFS-2400 Update to new ceph and aws libraries
VFS-2524 Improve translation of acl and xattr records.
VFS-2524 Add basic attributes to /attributes endpoint.
VFS-2524 Fix wrong file owner in cdmi.
VFS-2524 Add copy operation to cdmi interface.
VFS-2573 Repair custom metadata propagation
VFS-2659 Rework user and group models
VFS-2625 Fix public share view not retrieving fiels correctly
VFS-2524 Add move operation to cdmi, split move and copy tests.
VFS-2594 Add remove_metadata operation.
VFS-2180 Implement support for read only spaces
VFS-2456 Add metadata to public view
VFS-2456 Implement first version of metadata backend
VFS-2555 Add shares field to file attr.
VFS-2405 Adjust to new shares API in OP
VFS-2555 Improve share permissions and guest user management.
VFS-2472 Convert metadata to from proplists to maps.
VFS-2472 Unify file identifiers in REST interface.
VFS-2472 Add listing and getting inherited xattrs to REST API.
VFS-2309 oz test mock updated to match actual implementation
VFS-2311 Add private RPC to retrieve file download URL
VFS-2189 Close connection after file upload failure
VFS-2303 Add metadata-id endpoint.
VFS-2303 Add filters for getting metadata.
VFS-2303 Add query-index rest endpoint.
VFS-2269 Enable Symmetric Multiprocessing
VFS-2303 Adjust metadata changes stream to the new metadata organization.
VFS-2319 Reimplement monitoring using events
VFS-2303 Add basic metadata operations.
VFS-2049 Make file_consistency work after system restart.
VFS-2049 Improve file_consistency model.
VFS-2303 Add support for rdf metadata.
VFS-2361 Turn off HSTS by default, allow configuration via app.confi
The list of open and closed issues can be found also in GitHub:
Update procedure
There is no way to upgrade from previous release, the clean installation is needed.
Documentation on clean installation is available at:
Recomended operating system for all Onedata services is Ubuntu 16.04.
Oneclient:
any platform that supports Docker Engine (>1.11)
Ubuntu 14.04, 15.10, 16.04
Centos 7
Fedora 23
Oneprovider:
any platform that supports Docker Engine (>1.11)
Ubuntu 15.10, 16.04
Fedora 23
Onezone:
any platform that supports Docker Engine (>1.11)
Ubuntu 15.10, 16.04
Fedora 23
Artefacts available from IDNIGO-DataCloud repositories:
CentOS 7:
Ubuntu 14.04:
Docker images:
indigodatacloud/onezone:3.0.0.11
indigodatacloud/oneprovider:3.0.0.11
indigodatacloud/oneclient:3.0.0.11
The current update provides improved TOSCA templates parsing and IAM access token handling.
Fixes:
Improvements:
This release require a parameter change regarding the zabbix wrapper endpoint - the environment variable provided during startup must be changed from: </br> http://${host}:${port}/monitoring/adapters/zabbix/zones/indigo/types/*service*/groups/Cloud_Providers/hosts/</br> to </br> http://${host}:${port}/monitoring/adapters/zabbix/zones/indigo/types/*infrastructure*/groups/Cloud_Providers/hosts/</br>
Thus, the upgrade operations are:
Stop the old container:</br>
sudo docker stop orchestrator</br>
Remove the old container:</br>
sudo docker rm orchestrator</br>
Pull the new image version:
sudo docker pull indigodatacloud/orchestrator:1.2.1-FINAL</br>
Start the new version:</br>
docker run ...*updated parameters*... indigodatacloud/orchestrator:1.2.1-FINAL</br>
Docker Container:
This update brings many new features and bug fixes like
new Synergy Service features:
added support for OpenStack DOMAIN to shell.py
use pbr fully for easier package building
new Synergy Schedule Manager features:
OpenStack projects can now access to two distinct quota kinds: private and shared quota (see documentation)
Synergy CLI command enhanced in order to be compliant with the OpenStack style
KeystoneManager enhanced with the support of trust tokens; added the new "clock skew" feature needed for renewing the tokens a delta time before their expiration (new 'clock_skew' parameter)
SchedulerManager enhanced with the implementation of the BACKFILL scheduling algorithm (new 'backfill_depth' parameter)
NovaManager enhanced with the support to METADATA (new 'metadata_proxy_shared_secret' parameter)
use pbr fully for easier package building
breaking changes
Synergy doesn't rely anymore on nova.conf
Synergy CLI changed (see documentation)
Changes for synergy-service since v1.0.1
Features
Added support for OpenStack DOMAIN to shell.py
use pbr fully for easier package building
Bug fixes
Replaces uuid.uuid4 with uuidutils.generate_uuid
[packaging] make docker aware of PKG_VERSION
Update changelogs and system package versions
Clean up oslo imports
Update the Sphinx documentation
fix packaging with docker and its documentation
Distribute tabulate as part of Synergy
Remove versions for required packages
fix missing "requests" from the requirements
Updated coverage configuration file
fix docker packaging for CentOS
fix wrong version of eventlet
fix docs for packaging with Ubuntu
fix to get the synergy version when packaging
fix required packages when packaging
RPM: don't output errors on uninstallation
Fix conf for AMQP virtual host
Added unit tests
Fixed destroy() method
Fixed serializer
Fixed logging for managers
fix eventlet and dateutil required versions
Fix requirement version pinning
setup.cfg mod
setup.cfg updated
Use dependency pinning
Streamline packaging with docker
Cleanup tox.ini: Remove obsolete constraints
improved serialization
Managers are now serializable
improved command shell by using the 'tabulate' module
listManagers, startManager, stopManager and getManagerStatus methods now use the Managers serialization
'tabulate' module dependence added
updated unit tests
changed entry points
Changes for synergy-scheduler-manager since v1.0.1
Breaking changes
Synergy doesn't rely anymore on nova.conf
Features
Add the new clock_skew parameter for KeystoneManager
Add a backfill_depth parameter
CLI command "synergy usage show" enhanced
Scheduler managers enhanced
use pbr fully for easier package building
Bug fixes
Make SchedulerManager handle ERROR notifications
fix: update required version of synergy-service
[packaging] make docker aware of PKG_VERSION
Synergy releases in advance the VMs that are going to be destroyed
Queue.updatePriority() takes much time if the queue is large
Invalid input for field/attribute quota_class_set
KeystoneManager.authenticate() uses a wrong domain attribute
Item readjustment fixed in PriorityQueue
The FairshareManager should not use the Manager.condition var
Fix scheduling when shared quota is disabled
Update changelogs and system package versions
Clean up oslo imports
Method deserialize() fixed
The QuotaCommand shows a wrong value (%) for the field 'share'
Remove versions for required packages
fix git and pbr when packaging with docker
fix synergy-service version in spec file (rpm)
fix required packages when packaging
fix to get the synergy version when packaging
fix connection URL to RabbitMQ
Added Queue class to synergy_scheduler_manager/common/queue.py
Common objects and relative test units added
Destroy() method fixed
Fix requirement version pinning
Use dependency pinning
fix OpenStack CentOS repo for docker packaging
Cleanup tox.ini: Remove obsolete constraints
Add python-nova as a dependency
Packages:
CentOS7
Ubuntu 14.04
If the installation was done following the the instructions available in the , please follow the following steps to upgrade:
Service Reference Card is available
- Token validation across portal failure
For more details on installation and upgrade see
(8 open/6 closed)
(0 open/0 closed)
(0 open/0 closed)
(0 open/3 closed)
A repository supporting optional Operating Systems is available at providing packages for Fedora 23, Ubuntu 15.10 and 16.10
- Check access token expiration date and signature
- Update default monitoring endpoint
- Update custom INDIGO TOSCA types
- Improve DB connection handling during shutdown
now by default, the VMs are instantiated into the private quota. To select the shared quota, is needed to place special keys in the local user data file and pass it through the--user-data parameter at instance creation (see user guide )
Complete list of issues is available at:
Service Reference Card:
Instructions on how to Update an already existing service are avalable