Model-based reinforcement learning (MBRL) is widely recognized with the potential to be significantly more sample efficient than model-free reinforcement learning. How an accurate model can be developed automatically and efficiently from raw sensory inputs (such as images), especially for complex environments and tasks, is a challenging problem that hinders the broad application of MBRL in the real world. Recent developments in differentiable physics-based simulation and rendering provide a potential direction. In this talk, I will introduce a learning framework called SAGCI that leverages differentiable physics simulation to model the environment. It takes raw point clouds as inputs and produces an initial model represented as a Unified Robot Description Format (URDF) file, which is loaded into the simulation. The robot then utilizes interactive perception to online verify and modify the model. We propose a model-based learning algorithm combining object-centric and robot-centric stages to produce policies to accomplish manipulation tasks. Next, I will present a sensing-aware model-based reinforcement learning system called SAM-RL, combining differentiable physics simulation and rendering. SAM-RL automatically updates the model by comparing the rendered images with real raw images and produces the policy efficiently. With the sensing-aware learning pipeline, SAM-RL allows a robot to select an informative viewpoint to monitor the task process. We apply our framework to real-world experiments for accomplishing three manipulation tasks: robotic assembly, tool manipulation, and deformable object manipulation. I will close this talk by discussing the lessons learned and interesting open questions that remain.