Probabilistic spin glass - Conclusion

Marton Trencseni - Mon 31 January 2022 • Tagged with entropy, physics, spin, glass

I summarize the 5 previous posts on probabilistic spin glasses.

Continue reading

Probabilistic spin glass - Part V

Marton Trencseni - Thu 06 January 2022 • Tagged with entropy, physics, spin, glass

I use Monte Carlo simulations to explore the dynamic behaviour of probabilistic spin glasses, specifically how saturation scales with $p$ and $N$.

Continue reading

Probabilistic spin glass - Part IV

Marton Trencseni - Fri 31 December 2021 • Tagged with entropy, physics, spin, glass

This is a continuation of the previous articles on probabilistic spin glasses. I run simulations to understand the scaling behaviour for large spin glasses.

Continue reading

Probabilistic spin glass - Part III

Marton Trencseni - Sat 25 December 2021 • Tagged with entropy, physics, spin, glass

I run simulations to understand the dynamic probabilistic evolution of these toy models.

Continue reading

Probabilistic spin glass - Part II

Marton Trencseni - Sat 18 December 2021 • Tagged with entropy, physics, spin, glass

This is a continuation of the previous article on probabilistic spin glasses, with improvements to the simulation code and improved entropy computation.

Continue reading

Probabilistic spin glass - Part I

Marton Trencseni - Sat 11 December 2021 • Tagged with entropy, physics, spin, glass

I run Monte Carlo simulations on probabilistic spin glasses, a simple mathematical model of magnetized matter with short range interactions. I use entropy to characterize the model's order-disorder transition.

Continue reading

The physical Sackur-Tetrode entropy of an ideal gas

Marton Trencseni - Mon 29 November 2021 • Tagged with entropy, physics

I derive the Sackur-Tetrode equation for entropy of a monatomic ideal gas.

Continue reading

Entropy of an ideal gas with coarse-graining

Marton Trencseni - Fri 19 November 2021 • Tagged with entropy, physics

I show the first steps of how to arrive at a definition of entropy for a monatomic ideal gas modeled as hard billiard balls.

Continue reading

Entropy in Data Science

Marton Trencseni - Sun 24 October 2021 • Tagged with entropy

I discuss 4 uses of entropy in Data Science: (i) cross entropy as a loss function for training neural network classifiers (ii) entropy as a splitting criterion for building decision trees (iii) entropy for evaluating clustering algorithms (iv) entropy for understanding relationships in tabular data.

Information gain

Continue reading

Cross entropy, joint entropy, conditional entropy and relative entropy

Marton Trencseni - Sat 09 October 2021 • Tagged with entropy, cross-entropy, joint-entropy, conditional-entropy, relative-entropy, kullbackÔÇôleibler-diverence

What's the difference between cross entropy, joint entropy, conditional entropy and relative entropy?

Information gain

Continue reading

What's the entropy of a fair coin toss?

Marton Trencseni - Sat 25 September 2021 • Tagged with entropy, interviews, cross-entropy, physics

What's the entropy of a fair coin toss? What if the coin almost always returns Heads? My recruiter reports that very few candidates can answer these entropy related DS screening questions.

Entropy

Continue reading