Answer by John Jiang for Relation between cross entropy and conditional entropy
Conditional entropy is probably best viewed as the difference between two cross entropies: $H(Y|X) = H_{(X,Y)}(X,Y)-H_X(X)$. That is, it’s the incremental entropy from the probability given by X to...
View ArticleAnswer by leonbloy for Relation between cross entropy and conditional entropy
There is little or no relationship. The cross entropy relates only to the marginal distributions, (the dependence between $X$ and $Y$ do not matter) while the conditional entropy relates to the joint...
View ArticleRelation between cross entropy and conditional entropy
Is there a relationship between cross-entropy and conditional entropy between two categorical variables?Definition of cross-entropy:$$H_X(Y) = -\sum_{x} P(X=x)\log P(Y=x)$$Definition of conditional...
View Article
More Pages to Explore .....
