Data Foundry and Expert Models Platform Updates and New Features
Written by Creme Global
The past year has been very productive on the Expert Models and Data Foundry platform, with 12 major releases consisting of a combination of new features, updates to existing features, bug fixes and usability improvements. In this article, we go over some most notable improvements. Our primary focus was improving the user experience and making the system more robust, versatile and practical for our clients. All of the improvements below are on features that are available in both the Expert Models and Data Foundry platforms.
Working with data in tables is core to the Data Foundry and Expert Models workflow and this area has seen a number of targeted improvements aimed at increasing the range of what can be achieved in table view and streamlining the process.
New Data Types Available
Users can now specify the data type for new columns added to a table. This new feature allows users to simply and quickly create their own bespoke structured tables utilising our available field types.
Data types available for selection are:
- – Text (can store text strings),
- – Decimal (allows fractional numbers),
- – Integer (allows whole numbers only),
- – Boolean (a logic field that stores 1 or 0 / TRUE or FALSE),
- – Numeric Distribution (allows probabilistic expressions),
- – Date (used to represent your date information),
- – Datetime (used to store timestamp data), and
- – URL (use this to store web links/uniform resource identifiers).
Once a data type is set for a column, the data into these columns will be validated against the specified type.
Table Validation Improvements
Table validation is an important feature of Data Foundry and Expert Models. It is a standard feature which checks data in our structured tables and warns our users if data in a column is not valid based on the required type. Many of our scientific models in Expert Models will only allow you to select validated data as inputs for the model to protect the assessment from spurious data.
Users can now toggle on or off the table validation system while viewing or working on tables. This option gives you more freedom to work with your data, as validation may not be required in all scenarios. Once validation is paused, you won’t see validation errors popping up after every keystroke as you work. You can turn validation back on, if required, once you are finished editing your table if you want to check and validate your data.
On top of this new feature we have implemented an improved method of data validation for very large tables to preemptively prevent time-outs of table data validation attempts.
We have created a new Columns Selector feature, so our users can select a subset of the columns that they wish to view in a table. This feature is very useful for dealing with very wide tables, to narrow down the number of columns on display to a specific subset.
Saving Your Key Data Views
Working on large data files often requires users to create filters with which you can view an important subset of the data. We recognised the importance of these steps in the user workflow and created a new feature that enables users to save their filters. Users can now easily load previously created filters, instead of having to recreate them anew every time.
These saved filters can be saved for each individual table or saved for later use on all tables of a certain type (e.g. for all Subjects Tables, save a filter for ‘Females between the age of 18 and 35’).
Viewing data in graphs is a key feature of Data Foundry and Expert Models. Working on a large amount of data often requires users to create various graphs so that results can be easily visualised.
We have improved our graphing system with the addition of the cumulative histogram graph type as well as allowing log scale on numerical axes for our graphs.
We recognised the importance of setting up key graphs in the user workflow and have created a new feature that enables users to save their important graphs. Users can now easily load previously created graphs, instead of having to recreate their required graphs anew every time they want to view them.
These saved graphs can be saved for each individual table or saved for later use on all tables of a certain type and are made available to all users within your group.
Improved Data Uploading
Loading up of various files and tables onto the platform is key to the user workflow. Smooth and reliable upload functionality is therefore very important and we have made numerous improvements to how this is handled by Data Foundry and Expert Models.
When uploading files, our users now have a detailed information note that explains in-depth all system requirements for files being uploaded. Uploading system has been built up to support bulk uploading of a large number of files to the platform. Most commonly used file format for uploading, the CSV file, now prompts users to manually set the table fields to match the fields from the data being uploaded. On top of this Expert Models now supports uploading of significantly larger files – allowing even more flexibility in their workflow to our users.
These improvements are part of our ongoing improvements to the Data Foundry and Expert Models platform and are part of our roadmap:
- – creating secure data portals,
- – improving saved graphs feature
- – creating dashboard feature, and
- – various other enhancements.
As always detailed release notes for each platform update are available in the Support Centre inside the Expert Models platform. Users can simply log in to Expert Models, click on their profile icon and select Help followed by Expert Models Support Articles and Release Notes. These contain a detailed breakdown of all of our updates.