Assistant manager Carlos Compagnucci and assistant first-team coach Xavier Tamarit have also left the club.
The club are targeting a swift appointment - a third in less than two years - ideally before Sunday's FA Cup quarter-final at Wigan Athletic.
The club also conirmed that Pellegrino's backroom staff would be following the Argentinian through the exit door.
Southampton are now looking for their fifth manager since 2013.
The latest 3-0 defeat at relegation rivals Newcastle on Saturday proved to be the final straw for the Southampton hierarchy, with the club in danger of being relegated from the top flight for the first time since being promoted in 2012. However the team have struggled in front of goal, and he departs with Southampton one place and one point above the bottom three.
Southampton have a talented squad of players and will hope that a new manager can drive the side up the table in the last eight games - everything is up for grabs with Swansea, West Ham and Bournemouth all still to play before the season comes to a conclusion. The south-coast club made a decision to act two days after a 3-0 loss at Newcastle, after which Pellegrino said some of his players "gave up".
Silva has been linked with Portguese giants Benfica for next season, but it remains to be seen if he could turn down the chance to return to the Premier League.
"We have to say sorry to our fans because we didn't represent the values of the club", Pellegrino concluded.