It is permissible to take someone else to the house of one who you are certain will approve of that and will not mind. It is recommended to gather to eat
وَحَدَّثَنِي إِسْحَاقُ بْنُ مَنْصُورٍ، أَخْبَرَنَا أَبُو هِشَامٍ، - يَعْنِي الْمُغِيرَةَ بْنَ سَلَمَةَ - حَدَّثَنَا عَبْدُ الْوَاحِدِ بْنُ زِيَادٍ، حَدَّثَنَا يَزِيدُ، حَدَّثَنَا أَبُو حَازِمٍ، قَالَ سَمِعْتُ أَبَا هُرَيْرَةَ، يَقُولُ بَيْنَا أَبُو بَكْرٍ قَاعِدٌ وَعُمَرُ مَعَهُ إِذْ أَتَاهُمَا رَسُولُ اللَّهِ صلى الله عليه وسلم فَقَالَ " مَا أَقْعَدَكُمَا هَا هُنَا " . قَالاَ أَخْرَجَنَا الْجُوعُ مِنْ بُيُوتِنَا وَالَّذِي بَعَثَكَ بِالْحَقِّ . ثُمَّ ذَكَرَ نَحْوَ حَدِيثِ خَلَفِ بْنِ خَلِيفَةَ .
Abu Huraira reported: One day while Abu Bakr was sitting and there was with him Umar also there came to them Allah's Messenger (ﷺ) and he said: What makes you stay here? They said: It is hunger that has brought us out from our houses. By Him Who has stint you with Truth; the rest of the hadith is the same.