To optimize error we will require gradient of the error function. Which is straightforward application of derivatives.
\begin{align}
\text{sigmoid}'(x)&=\frac{e^{-x}}{(1+e^{-x})^2}\\
&=\text{sigmoid}(x)\text{ sigmoid}(1-x)
\end{align}
The error function dependent on the weights and bias. That is it is a scalar valued function from $\mathbb{R}^{n+1}$ to $\mathbb{R}$.
\begin{equation}
E(W, b)=-\frac{1}{m}\sum_{1}^m(1-y_i)\ln(1-p_i)+y_i\ln(p_i)
\end{equation}
Where $W=(w_1, w_2, \cdots, w_n)$.
Note that $$\frac{\partial p_i}{\partial w_j}=\frac{\partial (\text{sigmoid}(Wx^{(i)}+b))}{\partial w_j}=(p_i)(1-p_i)\cdot x_j$$
To evaluate gradient of E we will differentiante the term $(1-y_i)\ln(1-p_i)+y_i\ln(p_i)$, call it $E_i$, with repect of $w_j$.
\begin{align}
\frac{\partial E_i}{\partial w_j}&=\frac{\partial E_i}{\partial w_j}(1-y_i)\ln(1-p_i)+y_i\ln(p_i)\\
&=-(1-y_i)p_i\cdot x_j+y_i(1-p_i)\cdot x_j\\
&= (y_i-p_i)x_j
\end{align}
Note: $x_j$ is the entry from the i^th row and j^th column.
Finally,
\begin{equation}
\nabla E=-\sum_{i}^m(y_i-p_i)\cdot x_j
\end{equation}
We use this for gradient descent algorithm (which is also used in quadratic minimzation problem).
The above gradient can be written in matrix form
\begin{equation}
\nabla E=-(Y-P)\cdot X
\end{equation}
Let W denote the initialised weights.
Algorithm
W = W - learnrate*(grad(E))
