In the article in front, the operation that we had studied TensorFlow drive function uses a method (see an article: Master TensorFlow quickly (3) , we will continue to learn TensorFlow today.
The article basically is the loss function of study control TensorFlow.
One, what is loss function
Loss function (Loss Function) it is the very serious content in machine study, it is value of magnanimity model output and the difference that the target is worth, serve as a kind of when evaluate model effect main index namely, loss function is smaller, the rash club gender that indicates a model is better.
2, how to use loss function
When the model trains in TensorFlow, passing loss function to tell TensorFlow to forecast result photograph to compare an object is good as a result bad still. Below a variety of circumstances, we can give out the example data that the model trains and target data, loss function is to be forecasted quite namely value and given the difference between target cost.
Commonly used in TensorFlow loss function will introduce below.
1, the loss function that returns to a model
Explain the loss function that returns to a model above all, returning to a model is to forecast successive because of variable. Introduce to go to the lavatory, define first forecast a result (- 1 to the equal difference alignment of 1) , end result (target cost is 0) , code is as follows:
Import Tensorflow As Tfsess=tf.Session()y_pred=tf.linspace(-1. , 1. , 100)y_target=tf.constant(0. 100)y_target=tf.constant(0..
Attention, when training a model actually, forecasting a result is the result worth that the model exports, example provides end result.
(1) L1 criterion loss function (namely function of absolute value loss)
L1 criterion loss function is the poor value that is worth to forecasting value and end seek absolute value, formula is as follows:
Means calls to be as follows in TensorFlow:
Loss_l1_vals=tf.abs(y_pred-y_target)loss_l1_out=sess.run(loss_l1_vals)
L1 criterion loss function is not flowing around target cost, can bring about a model cannot well convergent.
(2) L2 criterion loss function (namely function of Ou La loss)
L2 criterion loss function is to forecast value and target cost to differ the sum of squares of the value, formula is as follows:
Should take average to L2, turn mean square into the error (MSE, mean Squared Error) , formula is as follows:
Means calls to be as follows in TensorFlow:
Loss_mse_vals= Tf.reduce.mean(tf.square(y_pred - Y_target))loss_mse_out = Sess.run(loss_mse_vals of error of mean square of Loss_l2_vals=tf.square(y_pred - Y_target)loss_l2_out=sess.run(loss_l2_vals)# of # L2 loss)
L2 criterion loss function has very good curvature around target cost, closer from the target astringent is slower, it is very useful loss function.
L1, L2 criterion following plan institute show loss function:
(3) Pseudo-Huber loss function
Huber loss function often is used at returning to a problem, it is section function, formula is as follows:
From this formula can see differ when incomplete (the poor value that calculates value and target value, namely Y-f(x) ) very small when, loss function is L2 model number, when incomplete difference is big, it is the linear function that L1 model figures.
Peseudo-Huber loss function is the successive, flowing estimation of Huber loss function, successive around the target, formula is as follows:
This formula relies on parameter Delta, delta is bigger, criterion the linear part of both sides is abrupter.
Means calls mediumly to be as follows in TensorFlow:
Delta=tf.constant(0.25)loss_huber_vals = Tf.mul(tf.square(delta) , tf.sqrt(1. + – of Y_pred)/delta)) of Tf.square(y_target – 1. ) Loss_huber_out = Sess.run(loss_huber_vals)
The comparative plan of function of loss of L1, L2, Huber is as follows, among them the Delta of Huber takes 0.25, two 5 values:
2, the loss function that classifies a model
Classified loss function basically is used at evaluating to forecast classified result, redefine calculates a value (- 3 to the equal difference alignment of 5) be worth with the target (target cost is 1) , following:
Y_pred=tf.linspace(-3. , 5. , 100)y_target=tf.constant(1. ) Y_targets=tf.fill([100, ] , 1. 1..
(1) Hinge loss function
Hinge loss is commonly used at 2 classification problem, basically use evaluate vector machine algorithm, but also use sometimes evaluate nerve network algorithm, formula is as follows:
Means calls mediumly to be as follows in TensorFlow:
Loss_hinge_vals = Tf.maximum(0. , 1. – Tf.mul(y_target, y_pred))loss_hinge_out = Sess.run(loss_hinge_vals)
In the code above, target cost is 1, should forecast a value to leave 1 closer, criterion loss function is smaller, pursue as follows:
(2) entropy of two kinds of across (Cross-entropy) loss function
Alternate entropy comes from at information theory, it is classification extensive loss function is used in the problem. Alternate entropy depict the distance between two probability distribution, when two probability distribution are more adjacent, their alternate entropy is smaller also, give P of two probability distribution and Q, be apart from as follows:
To two kinds of problems, when P=y of a probability, criterion another probability Q=1-y, because this takes the place of to change brief the formula after is as follows:
Means calls mediumly to be as follows in TensorFlow:
Loss_ce_vals = Tf.mul(y_target, tf.mul((1 of Tf.log(y_pred)) –. – Y_target) , tf.log(1. – Y_pred))loss_ce_out = Sess.run(loss_ce_vals)
Cross-entropy loss function basically applies on 2 classification problem, forecast a value to be probability value, limits extraction a cost is [0, 1] , loss function pursues as follows:
(3) function of loss of Sigmoid across entropy
As similar as the entropy of two kinds of across above, just will calculate a value Y_pred value has transition through Sigmoid function, calculate again alternate entropy loss. Have in TensorFlow inside buy this function, call means to be as follows:
Loss_sce_vals=tf.nn.sigmoid_cross_entropy_with_logits(y_pred, y_targets)loss_sce_out=sess.run(loss_sce_vals)
Can be worth the input decrescent as a result of Sigmoid function a lot of, thereby flowing calculate a value, make Sigmoid across entropy is in when forecasting a value to leave target cost further, its losing growth is done not have so abrupt. With the comparison of entropy of two kinds of across the plan is as follows:
(4) function of loss of entropy of the across that increase advantageous position
Function of loss of entropy of the across that increase advantageous position is function of loss of Sigmoid across entropy increase advantageous position, it is right of the target increase advantageous position. Assume weight is 0.5, means calls mediumly to be as follows in TensorFlow:
Weight = Tf.constant(0.5)loss_wce_vals = Tf.nn.weighted_cross_entropy_with_logits(y)vals, y_targets, weight)loss_wce_out = Sess.run(loss_wce_vals)
(5) function of loss of Softmax across entropy
Function of loss of Softmax across entropy is the output result of normalization of action Yu Fei, classify computational loss in the light of single goal only.
Change output into probability distribution as a result through Softmax function, facilitate the input arrives inside alternate entropy thereby have consideration (alternate entropy asks the input is probability) , softmax defines as follows:
The alternate entropy before union defines formula, criterion formula of function of loss of Softmax across entropy is as follows:
Means calls to be as follows in TensorFlow:
Y_pred=tf.constant([[1. , - 3. , 10. ] ] Y_target=tf.constant([[0.1, 0.02, 0.88]])loss_sce_vals=tf.nn.softmax_cross_entropy_with_logits(y_pred, y_target)loss_sce_out=sess.run(loss_sce_vals)
Use at returning to relevant loss function, comparative plan is as follows:
3, summary
The face undertakes a summary to all sorts of loss function below, following watches are shown:
In be used actually, use error of MSE mean square to returning to problem classics regular meeting (L2 is taken average) computational loss, use function of loss of Sigmoid across entropy to classifying problem classics regular meeting.
When everybody is being used, even the model with actual setting, specific basis, choose use loss function, hope the article is helpful to you.
Next " master TensorFlow quickly " series article, return the wonderful content that can more explains TensorFlow, expect please.
