What is the point of TensorFlow for Windows

Artificial Intelligence

TensorFlow is an open source machine learning framework. It is basically a library for numerical calculations using data stream graphs. The graph nodes represent mathematical operations, while the edges of the graph represent multidimensional arrays of data (tensors) flowing between them.

Tools in TensorFlow

This flexible architecture makes it possible to provide computations on one or more CPUs or GPUs in a desktop PC, server or mobile device without having to rewrite any code. TensorFlow also includes TensorBoard, a Data visualization toolkit.

While data flow graphs can still be built with TensorFlow and later executed in sessions, version 2.0 fully supports the Eager execution mode. This is an imperative define-by-run interface that calculates operations immediately without building a graph. The eager execution mode supports automatic differentiation via the tf.GradientTape API. One of the improvements in tf.keras (see Keras below) is support for Eager Execution.

While several other APIs were dropped in TensorFlow 2.0, one is Conversion tool for existing code, in addition to a compatibility library.

Estimators are the most scalable and production-oriented model types for machine learning from TensorFlow. Businesses can use either the pre-made Estimators use from google or write your own. Estimators themselves are built on tf.keras.layers, which simplifies customization. It is usually more convenient to build models with estimators rather than TensorFlow's low-level APIs. Ready-made estimators make it possible to work on a higher conceptual level than with the basic TensorFlow APIs

Keras

TensorFlow still supports its original low-level API. However, tf.keras is now the preferred one High-level API, an implementation of the Keras API standard that includes TensorFlow-specific enhancements. "High" and "Low" refer to how deep and "hardware close" the API is. Low-level means that more detailed but also more complex settings can be made. At the high level, the functions are abstracted so that fewer options are available, but the API is easier to use.

Keras is a high-level API for neural networks. It is written in Python and can run on top of TensorFlow, CNTK, or Theano. Additional backends such as MXNet or PlaidML are supported by third parties.

Keras was designed with ease of use in mind. It should be modular, easy to expand, and written in Python. The API is said to be "developed for people and not machines" and follows best practices that are intended to reduce the cognitive load on operation.

Neural layers, cost functions, optimizers, initialization schemes, activation functions and regulation schemes are all independent modules in Keras. They can be combined to create new models. New models, in turn, can easily be added as new classes and functions. Models are defined in Python code rather than as separate model configuration files.

The main reasons for using Keras are the design principles followed, above all the focus on user-friendliness. It's easy to learn and models are easy to build. In addition, Keras takes advantage of a large user base, supports a wide range of deployment options, multiple GPUs, and distributed training. Google, Microsoft, Amazon, Apple, Nvidia, Uber and many others support the tool.

Pruning API

The Optimization toolGoogle's Pruning API is technically based on Keras. Therefore it should be easy to integrate into existing Keras projects. The tool is intended to optimize machine learning models as early as the training phase.

According to the name, the API trims ML models. During training, she evaluates the connections between the various levels of the model. Unimportant or irrelevant connections are removed from the network. This reduces both the memory capacity required when saving the model and the main memory required to run the model, as well as the required CPU operations.

Horovod

Horovod is one developed by Uber distributed training framework for TensorFlow, Keras and the open source program library PyTorch. Horovod is designed to make distributed deep learning quick and easy to use. It is based on ideas from Baidu's experimental implementation of the TensorFlow Ring-Allreduce Algorithm.

Uber originally tried to use Distributed TensorFlow with parameter servers. The engineers found that the Message Passing Interface (MPI) model was less complicated and required fewer code adjustments. Uber claims the Horovod system allows an AI model to be trained roughly twice as fast as a traditional TensorFlow implementation.

Horovod uses Open MPI (or another MPI implementation) to exchange messages between nodes, and Nvidia NCCL for its highly optimized version of Ring Allreduce. Horovod achieves 90 percent scaling efficiency for Inception-v3 and ResNet-101. For VGG-16, it achieves a scaling efficiency of 68 percent on up to 512 Nvidia Pascal GPUs.

In December 2018, Uber announced that it was giving the Horovod project under the aegis of the LF Deep Learning Foundation for open source AI software to the Linux Foundation.

The Tony Project

LinkedIn made the code for their Tony project public in late 2018. According to Serdar Yegulalp from InfoWorld, the open source tool is used to Manage deep learning jobs in TensorFlow and to scale. To do this, it uses the YARN (Yet Another Resource Negotiator) job planning system in Hadoop.

While there are already a few other planning tools out there, LinkedIn noted a few limitations. TensorFlow on Spark, for example, runs the framework on the Apache Spark job engine, but is therefore very closely linked to Spark. While TensorFlowOnYARN has the same basic features as Tony, it is not maintained and has no fault tolerance.

According to LinkedIn, Tony uses YARN's resource and task planning system to set up TensorFlow jobs in a Hadoop cluster. In addition, the tool should enable

  • Schedule GPU-based TensorFlow jobs through Hadoop;

  • request different types of resources (CPUs or GPUs);

  • Allocate memory differently for TensorFlow nodes;

  • Regularly save job results to the Hadoop Distributed File System (HDFS) and resume at a point if they are interrupted or crash.

Tony divides the work into three different internal components: a client, an application master, and one that does the job. The client receives incoming TensorFlow jobs. The master coordinates with the YARN resource manager how the job is to be provisioned in YARN. The executive component is what is actually running on the YARN cluster to process the TensorFlow job.

According to LinkedIn, Tony does not incur any noticeable overhead for TensorFlow as it resides in the layer that orchestrates distributed TensorFlow. Therefore it does not affect the actual execution of the TensorFlow job.

Tony also works with the TensorBoard app to visualize, optimize and debug TensorFlow.

Incl

Inkling is a commercial one High level programming language from Bonsai (now a Microsoft subsidiary), which makes it easier to build AI applications. According to Paul Krill from InfoWorld, it compiles down to the TensorFlow library. Inkling is meant to represent AI in a way that programmers can focus on teaching a system instead of having to focus on the low-level mechanics.

Inkling abstracts dynamic AI algorithms that normally require machine learning expertise. According to Bonsai, the language is descriptive and its syntax is reminiscent of a mixture of Python and SQL. The aim of the language is to make machine learning accessible to developers and engineers who have no ML background but want to use the technology in their respective specialist areas.

Tensor2Tensor

Google's open source project Tensor2Tensor (T2T) aims to support the Reduce workloadto configure a deep learning model for training. It is a Python based library for workflow optimization of TensorFlow training tasks. Developers can use it to specify the key elements in a TensorFlow model and define their relationships with one another.

According to InfoWorld editor Serdar Yegulalp, the key elements are:

  • Records: T2T already supports numerous data sets for training. New data records can be added to the individual workflow or to the core T2T project via pull request.

  • Problems and Modalities: These describe which task is being trained for (e.g. speech recognition or translation) and which data should be received for this as well as generated from it. For example, an image recognition system would receive image data and output text descriptions.

  • Models: Many popular models are already registered with T2T and more can be added.

  • Hyperparameters: A hyperparameter is a parameter that is used to control the ML training algorithm and whose value, unlike other parameters, is not learned in the actual training of the model. Different sets of settings can be created in T2T that control the training process. These can be changed or interconnected as required.

  • Trainer: The parameters that are passed to the actual training binary file can be specified separately.

T2T offers presets for each element. Numerous popular models and data sets are already included. This allows you to quickly start training by reusing or expanding them. What T2T does not provide is a broader context beyond TensorFlow of how the deep learning project should be organized. It "just" makes it easier to use the framework.

Ludwig

Another tool that the Simplify deep learning training should comes as open source from the Horovod makers Uber. With Ludwig, ML models should be able to be trained and tested without any programming effort.

According to Heise, the tool takes a data type-related approach to model development, with specific encoders and decoders for each data type. However, different encoders and decoders should also be able to be used for each type. The desired encoder, including hyperparameters, should be able to be specified directly in the model configuration file (in YAML format) without having to write code for it. In the current version, Ludwig offers encoders and decoders for common data types such as binary values, floating point numbers, categories, discrete sequences, images, texts and time series, which can be supplemented with pre-trained models if necessary.

TensorWatch

With TensorWatch, Microsoft Research published an open sourceDebugging tool for machine learning, which should also help with complex problems.

Often no log files are created during model training because, depending on the size of the data records, they can cause high storage costs. However, this does not provide a general overview of errors in the model.

TensorWatch should visualize interactive real-time debugging in Jupyter Notebook as well as offer custom UIs and the possibility of integration into the Python ecosystem. Jupyter Notebook is an open source web application that can be used to create and share documents that contain live code, formulas, visualizations, or text. As supported visualization types