What does U.S. Digital Sciences Corporation do?

We design and develop custom software, AI systems, and cloud infrastructure for businesses seeking digital transformation and operational efficiency.

Where is U.S. Digital Sciences located?

Our headquarters are in Port Jefferson Station, New York, serving clients across the United States.

What industries does USDSC serve?

We serve technology, healthcare, finance, telecom, logistics, and education sectors with scalable digital solutions.

Artificial Intelligence - US Digital Sciences Corporation | Computing research and development services

Deep Learning Demo

Artificial Intelligence Can even run in your browser, no gpu,no crazy hardware.

Utilizing ConvnetJS

Current Input State

Current Statistics

Artificial Intelligence

This demo showcases the power of the Deep Q-Learning algorithm, a cornerstone of modern Artificial Intelligence. In this specific simulation, we utilize a 2D agent equipped with a complex sensory system. Specifically, the agent possesses nine “eyes” oriented at various angles. Each eye detects three distinct variables within its visibility range: distance to walls, distance to green objects (poison), and distance to red objects (apples).

How the Agent Navigates and Learns

The agent navigates its environment by choosing from five distinct movement actions. Furthermore, the system operates on a reward-based feedback loop. When the agent consumes a red “apple,” it receives a positive reward. Conversely, eating a green “poison” object results in a negative penalty.

Initially, the agent moves randomly with no understanding of its surroundings. However, the training process typically takes about thirty minutes under current parameter settings. During this time, the Artificial Intelligence engine builds a sophisticated policy. Consequently, the agent learns to identify and avoid states that lead to low rewards. By analyzing historical data, it begins to prioritize actions that lead to optimal states instead.

(Takes ~10 minutes to train with current settings.)

Start / Stop

The Technical Impact of Deep Learning

This simulation effectively demonstrates how a neural network can approximate a “Q-function” to predict future rewards. In addition, this technology serves as the foundation for autonomous vehicles and robotics. Therefore, the ability to compile sensory information into actionable decisions without human interaction is a vital necessity for modern automation.

Ultimately, this agent does more than just move; it evolves. By bridging the gap between raw data and intelligent behavior, Deep Q-Learning proves that machines can master complex tasks through trial, error, and optimization. As a result, businesses can leverage these same principles to automate logistics, trading, and resource management.

var num_inputs = 27; // 9 eyes, each sees 3 numbers (wall, green, red thing proximity)
var num_actions = 5; // 5 possible angles agent can turn
var temporal_window = 1; // amount of temporal memory. 0 = agent lives in-the-moment :)
var network_size = num_inputs*temporal_window + num_actions*temporal_window + num_inputs;

// the value function network computes a value of taking any of the possible actions
// given an input state. Here we specify one explicitly the hard way
// but user could also equivalently instead use opt.hidden_layer_sizes = [20,20]
// to just insert simple relu hidden layers.
var layer_defs = [];
layer_defs.push({type:'input', out_sx:1, out_sy:1, out_depth:network_size});
layer_defs.push({type:'fc', num_neurons: 50, activation:'relu'});
layer_defs.push({type:'fc', num_neurons: 50, activation:'relu'});
layer_defs.push({type:'regression', num_neurons:num_actions});

// options for the Temporal Difference learner that trains the above net
// by backpropping the temporal difference learning rule.
var tdtrainer_options = {learning_rate:0.001, momentum:0.0, batch_size:64, l2_decay:0.01};

var opt = {};
opt.temporal_window = temporal_window;
opt.experience_size = 30000;
opt.start_learn_threshold = 1000;
opt.gamma = 0.7;
opt.learning_steps_total = 200000;
opt.learning_steps_burnin = 3000;
opt.epsilon_min = 0.05;
opt.epsilon_test_time = 0.05;
opt.layer_defs = layer_defs;
opt.tdtrainer_options = tdtrainer_options;

var brain = new deepqlearn.Brain(num_inputs, num_actions, opt); // woohoo