34 questions
1
vote
1
answer
56
views
Custom PyEnvironment time_step and time_step_spec do not match
I'm creating a custom PyEnvironment in TensorFlow Agents to simulate the track and field decathlon. I've managed to create a functioning environment in the sense that I can use _step and _reset, but ...
- 31
1
vote
0
answers
158
views
PyDriver.run ValueError: Only supports batched time steps with a single batch dimension
i trying training tf agent in my environment
but have problome when running PyDriver.run
code is here
env spec code
self._action_spec = array_spec.BoundedArraySpec(
shape=(), dtype=np.float64, ...
- 19
1
vote
0
answers
160
views
tf_agents changing underlying suite_gym reward function
I'm trying to modify the MountainCarContinuous-v0 environment from suite_gym() because training is getting stuck in a local minima. The default reward function penalizes large actions which are ...
- 377
1
vote
0
answers
163
views
Which policy to use after training RL agent
When running the Tensorflow agents notebook for the Soft Actor-Critic Minitaur, https://www.tensorflow.org/agents/tutorials/7_SAC_minitaur_tutorial, the following directories are created under /tmp:
+...
1
vote
1
answer
369
views
Error when saving model with tensorflow-agents
I am trying to save a model with tensorflow-agents. First I define the following:
collect_policy = tf_agent.collect_policy
saver = PolicySaver(collect_policy, batch_size=None)
and then save the model ...
- 10.2k
0
votes
1
answer
59
views
Benefit of storing state as a list/integer in tensorflow agents
In the environment tutorial of tensorflow agents (https://www.tensorflow.org/agents/tutorials/2_environments_tutorial), the state is stored as an integer. When the state is required, it is converted ...
- 819
2
votes
1
answer
287
views
PPOAgent + Cartpole = ValueError: actor_network output spec does not match action spec:
I'm trying to experiment with using tf_agents' PPOAgent in the CartPole-v1 environment, but I am recieving the following error upon declaring the agent itself:
ValueError: actor_network output spec ...
- 31
3
votes
1
answer
1k
views
tf_agents doesn't properly learn a simple environment
I successfully followed this official tensorflow tutorial for training an agent to solve the 'CartPole-v0' gym environment. I only diverged from the tutorial in that I did not use reverb, because it's ...
- 853
1
vote
1
answer
232
views
How to get probability vector for all actions in tf-agents?
I'm working on Multi-Armed-Bandit problem, using LinearUCBAgent and LinearThompsonSamplingAgent but they both return a single action for an observation.
What I need is the probability for all the ...
- 11
0
votes
1
answer
701
views
TF-Agents error: TypeError: The two structures do not match: Trajectory vs. Trajectory
I am building a PPO agent side by side with the TF-Agents DQN tutorial. The idea was checking the basics structures needed for a simple tf-agent to work, and adapting it to a PPO agent.
I am also ...
- 84
0
votes
1
answer
633
views
Error in Google Colab: SystemError: This interpreter version: '3.7.10' doesn't match with version of the interpreter ViZDoom was compiled with: 3.7.11
I am using Google Colab to run ViZDoom combined with TensorFlow (specifically, the TF-Agents library).
Most of the times when I start the Colab notebook with my code I get the following error:
...
- 84
2
votes
1
answer
532
views
How to give a tuple of actions to a TF-Agents environment within the step method?
I want to model my environment such that each action is made of 3 possible sub-actions.
I've defined the _action_spec of my tf_agents.environments.py_environment.PyEnvironment as:
self._action_spec = ...
- 370
1
vote
1
answer
715
views
Shape of _observation_spec and shape of _action_spec in the Tf-agents environments example
In the tensorflow documentation for TF-Agents Environments there is an example of an environment for a simple (blackjack inspired) card game.
The init looks like the following:
class CardGameEnv(...
- 6,592
1
vote
1
answer
469
views
What changes occur when using tf_agents.environments.TFPyEnvironment to convert a Python RL environment into a TF environment?
I noticed something weird happening when converting a Python environment into a TF environment using tf_agents.environments.TFPyEnvironment and I'd like to ask you what general changes occur.
To ...
- 119
0
votes
1
answer
286
views
Merging and splitting time and action steps from TF-agents
I am trying to use TF-agents in a simple multi-agent non-cooperative parallel game. To simplify, I have two agents, defined with TF-agents. I defined a custom gym environment that takes as input the ...
- 1,288