Global_step 在 Tensorflow 意味着什么？

这是来自 TensorFlow 网站的 教程代码,

  1. 有人能解释一下 global_step是什么意思吗？

    我在 Tensorflow 的网站上发现了 全局步骤用计数训练步骤，但是我不太明白它到底是什么意思。

  2. 还有，设置 global_step时数字0是什么意思？

    def training(loss,learning_rate):
tf.summary.scalar('loss',loss)
optimizer = tf.train.GradientDescentOptimizer(learning_rate)
        

# Why 0 as the first parameter of the global_step tf.Variable?
global_step = tf.Variable(0, name='global_step',trainable=False)


train_op = optimizer.minimize(loss, global_step=global_step)
    

return train_op

根据 Tensorflow 文件 Global _ step: 变量更新后增加1。这是否意味着一次更新后 global_step变成1？

60992 次浏览
小开
最佳答案

global_step refers to the number of batches seen by the graph. Every time a batch is provided, the weights are updated in the direction that minimizes the loss. global_step just keeps track of the number of batches seen so far. When it is passed in the minimize() argument list, the variable is increased by one. Have a look at optimizer.minimize().

You can get the global_step value using tf.train.global_step(). Also handy are the utility methods tf.train.get_global_step or tf.train.get_or_create_global_step.

0 is the initial value of the global step in this context.

小开

The global_step Variable holds the total number of steps during training across the tasks (each step index will occur only on a single task).

A timeline created by global_step helps us understand know where we are in the grand scheme, from each of the tasks separately. For instance, the loss and accuracy could be plotted against global_step on Tensorboard.

小开

show you a vivid sample below:

code:

train_op = tf.train.GradientDescentOptimizer(learning_rate=LEARNING_RATE).minimize(loss_tensor,global_step=tf.train.create_global_step())
with tf.Session() as sess:
...
tf.logging.log_every_n(tf.logging.INFO,"np.mean(loss_evl)= %f at step %d",100,np.mean(loss_evl),sess.run(tf.train.get_global_step()))

corresponding print

INFO:tensorflow:np.mean(loss_evl)= 1.396970 at step 1
INFO:tensorflow:np.mean(loss_evl)= 1.221397 at step 101
INFO:tensorflow:np.mean(loss_evl)= 1.061688 at step 201
小开

There are networks, e.g. GANs, that may need two (or more) different steps. Training a GANs with the WGAN specification requires that the steps on the discriminator (or critic) D are more than the ones done on the generator G. In that case, it is usefull to declare different global_steps variables.

Example: (G_lossand D_loss are the loss of the generator and the discriminator)

G_global_step = tf.Variable(0, name='G_global_step', trainable=False)
D_global_step = tf.Variable(0, name='D_global_step', trainable=False)


minimizer = tf.train.RMSPropOptimizer(learning_rate=0.00005)


G_solver = minimizer.minimize(G_loss, var_list=params, global_step=G_global_step)
D_solver = minimizer.minimize(D_loss, var_list=params, global_step=D_global_step)

提交答案