Question: 1. The user clicks the button for my plugin.
2. Dialog pops up that has some textboxes, etc.
3. Inside that dialog, the user can click a button , and a color picker will pop up, letting the user choose which color he wants.

Answer: 
Today I had the same problem. Here is my solution, which is practically a copy-paste from the TableCell plugin from CKEditor 4.2. Hope it helps

```
CKEDITOR.dialog.add( 'myDialog', function( editor ) {
return {
    title: 'Add Data',
    minWidth: 300,
    minHeight: 200,
    contents: [
        {
            id: 'dataTab',
            label: 'Line',
            title: 'Line',
            elements: [
                ...
                {   
                    type: "hbox",
                    padding: 0,
                    widths: ["80%", "20%"],
                    children: [
                        {
                            id: 'linecolor',
                            type: 'text',
                            label: 'Line color',
                            setup: function( element ) {
                                ...
                            },
                            commit: function( element ) {
                                ...
                            }
                        },
                        {
                            type: "button",
                            id: "lineColorChooser",
                            "class": "colorChooser",
                            label: "Choose",
                            style: "margin-left: 8px",
                            onLoad: function () {
                                this.getElement().getParent().setStyle("vertical-align", "bottom")
                            },
                            onClick: function () {
                                editor.getColorFromDialog(function (color) {
                                    color && this.getDialog().getContentElement("dataTab", "linecolor").setValue( color );
                                    this.focus()
                                }, this)
                            }
                        }
                    ]
                },
                ...
            ]
        }
    ],

};
});

```
Use the following color picker plugin; attached to an `input[type="text"]` field in your CKEditor `form`:

<http://jscolor.com/>

This implementation is different in that the user does not need to `click` a button to select a color. The color selection is triggered when the user `focus`es into the appropriate field.