ASRC GOES 7:00 – 4:30, 6:30 – 7:00
7:00 – 3:00 ASRC GOES
import logging
import concurrent.futures
import threading
import time
def thread_function(name):
logging.info("Task %s: starting on thread %s", name, threading.current_thread().name)
time.sleep(2)
logging.info("Task %s: finishing on thread %s", name, threading.current_thread().name)
if __name__ == "__main__":
num_tasks = 5
num_gpus = 1
format = "%(asctime)s: %(message)s"
logging.basicConfig(format=format, level=logging.INFO,
datefmt="%H:%M:%S")
with concurrent.futures.ThreadPoolExecutor(max_workers=num_gpus) as executor:
result = executor.map(thread_function, range(num_tasks))
logging.info("Main : all done")
As you can see, it’s possible to have a thread for each gpu, while having them iterate over a larger set of tasks. Now I need to extract the gpu name from the thread info. In other words, ThreadPoolExecutor-0_0 needs to map to gpu:1.
import concurrent.futures
import threading
import time
from typing import List
import re
last_num_in_str_re = '(\d+)(?!.*\d)'
prog = re.compile(last_num_in_str_re)
def thread_function(args:List):
num = prog.search(threading.current_thread().name) # get the last number in a string
gpu_str = "gpu:{}".format(int(num.group(0))+1)
print("{}: starting on {}".format(args["name"], gpu_str))
time.sleep(2)
print("{}: finishing on {}".format(args["name"], gpu_str))
if __name__ == "__main__":
num_tasks = 5
num_gpus = 5
task_list = []
for i in range(num_tasks):
task = {"name":"task_{}".format(i), "value":2+(i/10)}
task_list.append(task)
with concurrent.futures.ThreadPoolExecutor(max_workers=num_gpus) as executor:
result = executor.map(thread_function, task_list)
print("Finished Main")
And that gives me:
task_0: starting on gpu:1 task_1: starting on gpu:2 task_0: finishing on gpu:1, after sleeping 2.0 seconds task_2: starting on gpu:1 task_1: finishing on gpu:2, after sleeping 2.1 seconds task_3: starting on gpu:2 task_2: finishing on gpu:1, after sleeping 2.2 seconds task_4: starting on gpu:1 task_3: finishing on gpu:2, after sleeping 2.3 seconds task_4: finishing on gpu:1, after sleeping 2.4 seconds Finished Main
So the only think left is to integrate this into TimeSeriesMl2
7:00 – 5:00 ASRC GOES
7:00 – 3:00 ASRC GOES
strategy = tf.distribute.OneDeviceStrategy(device="/gpu:0")
with strategy.scope():
model = tf.keras.Sequential()
# Adds a densely-connected layer with 64 units to the model:
model.add(layers.Dense(sequence_length, activation='relu', input_shape=(sequence_length,)))
# Add another:
model.add(layers.Dense(200, activation='relu'))
model.add(layers.Dense(200, activation='relu'))
# Add a layer with 10 output units:
model.add(layers.Dense(sequence_length))
loss_func = tf.keras.losses.MeanSquaredError()
opt_func = tf.keras.optimizers.Adam(0.01)
model.compile(optimizer= opt_func,
loss=loss_func,
metrics=['accuracy'])
noise = 0.0
full_mat, train_mat, test_mat = generate_train_test(num_funcs, rows_per_func, noise)
model.fit(train_mat, test_mat, epochs=70, batch_size=13)
model.evaluate(train_mat, test_mat)
model.save(model_name)
And here’s an example of predicting
strategy = tf.distribute.OneDeviceStrategy(device="/gpu:0")
with strategy.scope():
model = tf.keras.models.load_model(model_name)
full_mat, train_mat, test_mat = generate_train_test(num_funcs, rows_per_func, noise)
predict_mat = model.predict(train_mat)
# Let's try some immediate inference
for i in range(10):
pitch = random.random()/2.0 + 0.5
roll = random.random()/2.0 + 0.5
yaw = random.random()/2.0 + 0.5
inp_vec = np.array([[pitch, roll, yaw]])
eff_mat = model.predict(inp_vec)
print("input: pitch={:.2f}, roll={:.2f}, yaw={:.2f} efficiencies: pitch={:.2f}%, roll={:.2f}%, yaw={:.2f}%".
format(inp_vec[0][0], inp_vec[0][1], inp_vec[0][2], eff_mat[0][0]*100, eff_mat[0][1]*100, eff_mat[0][2]*100))
7:00 – 4:00 ASRC GOES
Chris Brew, Facebook. christopher.brew[at]gmail.com
Giovanni Da San Martino, Qatar Computing Research Institute. gmartino[at]qf.org.qa
Anna Feldman, Montclair State University. feldmana[at]montclair.edu
Chris Leberknight, Montclair State University. leberknightc[at]montclair.edu
Preslav Nakov, Qatar Computing Research Institute. pnakov[at]qf.org.qa
7:00 – ASRC GOES
\makeatletter
\def\set@curr@file#1{%
\begingroup
\escapechar\m@ne
\xdef\@curr@file{\expandafter\string\csname #1\endcsname}%
\endgroup
}
\def\quote@name#1{"\quote@@name#1\@gobble""}
\def\quote@@name#1"{#1\quote@@name}
\def\unquote@name#1{\quote@@name#1\@gobble"}
\makeatother
7:00 – 10:00 ASRC FAA
ASRC GOES – 7:00 – 4:00
7:00 – 4:00 ASRC GEOS
7:00 – ASRC GOES
City Arts & Lectures: Privacy and Technology
7:00 – 8:00 ASRC GOES
Chores calls!
Trump, Qanon and an impending judgment day: Behind the Facebook-fueled rise of The Epoch Times
By the numbers, there is no bigger advocate of President Donald Trump on Facebook than The Epoch Times. The small New York-based nonprofit news outlet has spent more than $1.5 million on about 11,000 pro-Trump advertisements in the last six months, according to data from Facebook’s advertising archive — more than any organization outside of the Trump campaign itself, and more than most Democratic presidential candidates have spent on their own campaigns. Those video ads — in which unidentified spokespeople thumb through a newspaper to praise Trump, peddle conspiracy theories about the “Deep State,” and criticize “fake news” media — strike a familiar tone in the online conservative news ecosystem. The Epoch Times looks like many of the conservative outlets that have gained followings in recent years.
7:00 – 4:00 ASRC GOES
ASRC GOES 7:00 – 4:00
Data Dictionary as a spreadsheet
7:00 – 4:00 ASRC GOES
7:00 – 5:00 ASRC GOES
viztales 
You must be logged in to post a comment.