From Automations to Streams to engine performance, FME 2021.0 provides a new quality of working with data. See what you can gain thanks to the new version!
FME 2021.0 brings a lot of new possibilities: major developments have been in the works to help you connect with even more systems, transform with speed and agility, and automate with increased control. Here are the areas where new functionalities have been added:
Expanded System Support
Enhanced User Experience
Advanced Automations
Autoscaling
Expanded System Support: Connecting More Data
To continue expanding your data reach, we’ve added support for 10+ new systems to ensure that no matter where your data is gathered, stored, or used, FME can help you make your data accessible and usable for people who need it. To highlight a few new systems here:
- Apache Parquet (Reader/Writer) – Read and write datasets from this high data volume, column-oriented data storage, the future of CSV. This is common in big data workflows.
- PROJ7 (Transformer) – Access grids remotely in this new version of PROJ.
- RabbitMQ (Connector) – Connect to the RabbitMQ message broker (AMQP 0-9-1) to send, receive, acknowledge, or count messages to support data streaming workflows.
- CKAN (Connector) – Upload and download data from this popular open-source open data portal.
- BIM360 Docs (Connector) – Upload, download, or list files hosted on BIM360 Docs directly to and from FME.
- Esri CityEngine (FME Hub) – Generate 3D models from input geometry and CityEngine rules packages. This can be used for urban planning and automated maintenance of digital twins.
Enhanced User Experience: Easier and Faster Than Ever
In FME 2021.0 you’ll see significant performance enhancements that improve your experience with usability, flexibility, and speed. With Data-Aware Transformer Dialogues now available in the AttributeManager and a new visual Parameter Dialog Builder, you can build increasingly streamlined and reusable workflows, saving even more time. This is a huge plus for anyone using FME Hub to share workspaces and custom transformers. FME’s inner workings were fine-tuned to make transformers easier to configure, perform faster than ever, and process streaming data in real-time.
Streaming data of any volume, from low or sporadic to dedicated high volume where low latency is a requirement, are perfect for Streams. As an alternative to bulk or batch workflows, Streams enables data to be processed before it’s stored so you can summarize, filter, and use data the way you need to.
Advanced Automations: Increased Control and Robustness
If you work with Automations that chain together heavy job processing or need to troubleshoot complex triggers, a Manual Trigger is now available so you can run on-demand Automations with the click of a button or through a REST call.
With Streams, a brand new interface in FME Server, it’s easier than ever to integrate streaming and IoT data using event streams for continuous, real-time data processing without any coding. As an alternative to bulk or batch processing, stream processing enables you to connect and integrate streams, enrich and analyze data for real-time decision making, no matter the source and volume of your data.
Autoscaling
- How much CPU and memory resources do my workspaces require?
- Should I implement Dynamic Engines, and how should I assign jobs or queues to them?
- Job Execution Metrics – with access to metrics including job processing time, memory usage, CPU time, and CPU percent, you can identify when engines are most or least busy and use the statistics to power dashboards in other enterprise applications to track FME Server engine performance over time.
- Queue Control – a reimagined way to manage the load on FME Server engines. This intelligent feature helps admins set up rule-based queue assignments to maximize the usage of engines and the efficiency of job submissions on FME Server.
- Queued Time for Jobs – admins can access job queued time as API endpoints to better understand the processing capacity of FME engines and optimize their deployments accordingly.
With the groundwork set, FME’s developers will be working towards a more intelligent FME Server that returns historical, current, and predicted load information to help you make better decisions about when to scale engines.
Of course, it’s worth saying a few words about Dynamic Engines as they play a fundamental role in supporting Automations, Streams, and autoscaling. With the same capability as standard engines, Dynamic Engines enable organizations to use FME to solve new challenges in innovative and cost-effective ways.
Most of the innovations introduced are FME users’ ideas, which are reported on an ongoing basis when using the software to work with data. We therefore encourage you to submit any ideas yourself so that we can develop the FME platform together. You can do it here.
