Sfrgttk

Can't find the Shader/UVs tab

Making a sword in the stone, in a medieval world without magic

Does splitting a potentially monolithic application into several smaller ones help prevent bugs?

How to pass a string to a command that expects a file?

infinitive telling the purpose

Peter's Strange Word

Are babies of evil humanoid species inherently evil?

Do I really need to have a scientific explanation for my premise?

Placing subfig vertically

My story is written in English, but is set in my home country. What language should I use for the dialogue?

What wound would be of little consequence to a biped but terrible for a quadruped?

Offered promotion but I'm leaving. Should I tell?

In the late 1940’s to early 1950’s what technology was available that could melt a LOT of ice?

The bar has been raised

How are such low op-amp input currents possible?

They call me Inspector Morse

Single word request: Harming the benefactor

Virginia employer terminated employee and wants signing bonus returned

Does "variables should live in the smallest scope as possible" include the case "variables should not exist if possible"?

Force user to remove USB token

Why does Captain Marvel assume the planet where she lands would recognize her credentials?

2×2×2 rubik's cube corner is twisted!

Should I tell my boss the work he did was worthless

What is wrong with Escaped Shapeshifter's original wording?

What are the best books to study Neural Networks from a purely mathematical perspective?

Modern Mathematical Theory for Neural Networks, Cellular Automata, NeuroscienceTwo-layer Perceptron for XORdelay time from mathematical perspectiveReference Request: Neural Networks as Mathematical ModelsSources to learn and understand advanced probability in ML modelsHow can logicians benefit from development of artificial neural networks?Deriving diagonal approximation of Hessian in a neural networkNeural Networks and the Chain RuleWhat are some current best practices for function approximation using neural networks?What does oplus symbol ⊕ do for 2 images in Convolution Neural Networks

5

3

$begingroup$

I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).

Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?

matrices book-recommendation mathematical-modeling neural-networks

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

5

3

$begingroup$

I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).

Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?

matrices book-recommendation mathematical-modeling neural-networks

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

$begingroup$

I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).

Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?

matrices book-recommendation mathematical-modeling neural-networks

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 2 hours ago

EliEli

341

2 Answers
2

active

oldest

votes

3

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 2 hours ago

answered 2 hours ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

add a comment | 

2

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
  $endgroup$
  – Eli
  2 hours ago
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

Eli is a new contributor. Be nice, and check out our Code of Conduct.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3146016%2fwhat-are-the-best-books-to-study-neural-networks-from-a-purely-mathematical-pers%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

3

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 2 hours ago

answered 2 hours ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

add a comment | 

3

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 2 hours ago

answered 2 hours ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

add a comment | 

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 2 hours ago

answered 2 hours ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

share|cite|improve this answer

edited 2 hours ago

answered 2 hours ago

Shamisen ExpertShamisen Expert

2,81821945

answered 2 hours ago

Shamisen ExpertShamisen Expert

2,81821945

answered 2 hours ago

Shamisen ExpertShamisen Expert

2,81821945

2

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
  $endgroup$
  – Eli
  2 hours ago
  

  
  
  
  

add a comment | 

2

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
  $endgroup$
  – Eli
  2 hours ago
  

  
  
  
  

add a comment | 

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144