During an intraoral scan session, a processing device receives a plurality of intraoral images of a dental site. The processing device determines that a historical template of the dental site is existent and registers a first intraoral image of the plurality of intraoral images to the historical template at a first region of the dental site depicted in the historical template. The processing device may register one or more remaining intraoral images of the plurality of intraoral images to the historical template at a second region of the dental site depicted in the historical template. The first region may be separated from the second region.
Embodiments of the present invention relate to the field of intraoral scanning and, in particular, to a system and method for improving the efficiency and accuracy of intraoral scanning.
In both prosthodontic and orthodontic procedures, obtaining a three-dimensional (3D) model of a dental site in the oral cavity is an initial procedure that is performed. In orthodontic procedures it can be important to provide a model of one or both jaws. 3D models created by the use of a 3D modeling scanner (e.g., a scanning wand) aid in the accurate creation of molds for corrective work. Scans may typically be performed when a patient comes in for a progress update check, to receive additional treatment, or sometimes during a routine hygiene appointment. Historically, each scan for a particular patient is treated as a new scan, unrelated to past scans of the same patient.
Additionally, scans have historically been cumbersome and time consuming. 30 scans rely on being able to stitch (hereafter referred to as "register") images of a scan session captured by a scanner to generate an accurate and complete 3D model. Frequently, images of a scan session may not be registered due to insufficient overlap between images. Images that cannot be registered to another image of the scan session are discarded, and a user interrupts the session to rescan any discarded areas. Furthermore, many scanners have a difficult time initializing after an interruption due to insufficient overlap between images or excessive wand movement. Such limitations are frustrating and impose additional time to work through.
FIG. 1 illustrates one embodiment of a system for performing intraoral scanning and generating a virtual three dimensional model of a dental site.
FIG. 2A illustrates a flow diagram for a first method of referencing a historical scan during an intraoral scan session, in accordance with embodiments of the present invention.
FIG. 2B illustrates a flow diagram for a second method of referencing a historical scan during an intraoral scan session, in accordance with embodiments of the present invention.
FIG. 3A illustrates a flow diagram for a method of generating a surface of an intraoral scan session, in accordance with embodiments of the present invention.
FIG. 3B illustrates a flow diagram for a method of determining a depiction of a change to a dental site between a current intraoral scan session and a past intraoral scan session, in accordance with embodiments of the present invention.
FIG. 4 illustrates a flow diagram for another method of referencing a historical scan during an intraoral scan session, in accordance with embodiments of the present invention.
FIG. 5 illustrates a three dimensional representation of intraoral images taken during a current scan session that have been registered to a historical template, in accordance with embodiments of the present invention.
FIG. 6 illustrates a block diagram of an example computing device, in accordance with embodiments of the present invention.
Described herein is a method and apparatus for improving the quality of scans, such as intraoral scans taken of dental sites for patients. During a scan session, a user (e.g., a dental practitioner) of a scanner may generate multiple different images (also referred to as scans) of a dental site, model of a dental site, or other object. The images may be discrete images (e.g., point-and-shoot images) or frames from a video (e.g., a continuous scan). These images may not capture all of the areas of the dental site and/or there may be areas for which there is conflicting data between images. For example, images recorded during a scan session may not be adequately registered to other images, thus yielding an incomplete arrangement of images of the scan session. Historically, images that cannot or should not be registered to other images in the scan session may be discarded, leaving the user to rescan the areas not captured. In embodiments described herein, such images are registered to a template that is based on a prior scan. This can facilitate quick and accurate scan sessions by reducing or eliminating discarded images. Additionally, during a scan session regions of a dental site that have been scanned in the scan session may be registered to the historical template and displayed in relation to the historical template in a three dimensional image. Current scan data may be represented using first imagery (e.g., solid surfaces) and historical scan data may be represented using second imagery (e.g., translucent surfaces or pixelated surfaces). As additional intraoral images are received during the scan session, these intraoral images may be registered to previously received intraoral images from the current scan session or to the historical template. The three dimensional image may then be updated to reflect the new intraoral images.
Embodiments described herein are discussed with reference to intraoral scanners, intraoral images, intraoral scan sessions, and so forth. However, it should be understood that embodiments also apply to other types of scanners than intraoral scanners. Embodiments may apply to any type of scanner that takes multiple images and registers these images together to form a combined image or virtual model. For example, embodiments may apply to desktop model scanners, computed tomography (CT scanners, and so forth. Additionally, it should be understood that the intraoral scanners or other scanners may be used to scan objects other than dental sites in an oral cavity. For example, embodiments may apply to scans performed on physical models of a dental site or any other object. Accordingly, embodiments describing intraoral images should be understood as being generally applicable to any types of images generated by a scanner, embodiments describing intraoral scan sessions should be understood as being applicable to scan sessions for any type of object, and embodiments describing intraoral scanners should be understood as being generally applicable to many types of scanners.
FIG. 1 illustrates one embodiment of a system 100 for performing intraoral scanning and/or generating a virtual three dimensional model of a dental site. In one embodiment, system 100 carries out one or more operations below described in method 200. System 100 includes a computing device 105 that may be coupled to a scanner 150 and/or a data store 110.
Computing device 105 may include a processing device, memory, secondary storage, one or more input devices (e.g., such as a keyboard, mouse, tablet, and so on), one or more output devices (e.g., a display, a printer, etc.), and/or other hardware components. Computing device 105 may be connected to a data store 110 either directly or via a network. The network may be a local area network (LAN), a public wide area network (WAN) (e.g., the Internet), a private WAN (e.g., an intranet), or a combination thereof. The computing device 105 may be integrated into the scanner 150 in some embodiments to improve performance and mobility.
Data store 110 may be an internal data store, or an external data store that is connected to computing device 105 directly or via a network. Examples of network data stores include a storage area network (SAN), a network attached storage (NAS), and a storage service provided by a cloud computing service provider. Data store 110 may include a file system, a database, or other data storage arrangement.
In some embodiments, a scanner 150 for obtaining three-dimensional (3D) data of a dental site in a patient's oral cavity is operatively connected to the computing device 105. Scanner 150 may include a probe (e.g., a hand held probe) for optically capturing three dimensional structures (e.g., by confocal focusing of an array of light beams). One example of such a scanner 150 is the iTero® intraoral digital scanner manufactured by Align Technology, Inc. Other examples of intraoral scanners include the 1M™ True Definition scanner, the Apollo DI intraoral scanner, the CEREC AC intraoral scanner manufactured by Dentsply Sirona®, and the Trios® intraoral scanner manufactured by 3Shape®.
The scanner 150 may be used to perform an intraoral scan of a patient's oral cavity. An intraoral scan application 108 running on computing device 105 may communicate with the scanner 150 to effectuate the intraoral scan. A result of the intraoral scan may be a sequence of intraoral images that have been discretely generated (e.g., by pressing on a "generate image" button of the scanner for each image). Alternatively, a result of the intraoral scan may be one or more videos of the patient's oral cavity. An operator may start recording the video with the scanner 150 at a first position in the oral cavity, move the scanner 150 within the oral cavity to a second position while the video is being taken, and then stop recording the video. In some embodiments, recording may start automatically as the scanner identifies either teeth or soft tissue from the oral cavity. The scanner 150 may transmit the discrete intraoral images or intraoral video (referred to collectively as intraoral image data 135) to the computing device 105. Computing device 105 may store the image data 135 in data store 110. Alternatively, scanner 150 may be connected to another system that stores the image data in data store 110. In such an embodiment, scanner 150 may not be connected to computing device 105.
According to an example, a user (e.g., a practitioner) may subject a patient to intraoral scanning. In doing so, the user may apply scanner 150 to one or more patient intraoral locations. The scanning may be divided into one or more segments. As an example the segments may include a lower buccal region of the patient, a lower lingual region of the patient, a upper buccal region of the patient, an upper lingual region of the patient, one or more preparation teeth of the patient (e.g., teeth of the patient to which a dental device such as a crown or an orthodontic alignment device will be applied), one or more teeth which are contacts of preparation teeth (e.g., teeth not themselves subject to a dental device but which are located next to one or more such teeth or which interface with one or more such teeth upon mouth closure), and/or patient bite (e.g., scanning performed with closure of the patient's mouth with scan being directed towards an interface area of the patient's upper and lower teeth). Via such scanner application, the scanner 150 may provide image data (also referred to as scan data) 135 to computing device 105. The image data 135 may include 2D intraoral images and/or 3D intraoral images. Such images may be provided from the scanner to the computing device 105 in the form of one or more points (e.g., one or more pixels and/or groups of pixels). For instance, the scanner 150 may provide such a 3D image as one or more point clouds.
The manner in which the oral cavity of a patient is to be scanned may depend on the procedure to be applied thereto. For example, if an upper or lower orthodontic appliance is to be created, then a lull scan of the mandibular or maxillary edentulous arches may be performed. In contrast, if a bridge is to be created, then just a portion of a total arch may be scanned which includes an edentulous region, the neighboring abutment teeth and the opposing arch and dentition. Thus, the dental practitioner may input the identity of a procedure to be performed into intraoral scan application 108. For this purpose, the dental practitioner may choose the procedure from a number of preset options on a drop-down menu or the like, from icons or via any other suitable graphical input interface. Alternatively, the identity of the procedure may be input in any other suitable way, for example by means of preset code, notation or any other suitable manner, intraoral scan application 108 having been suitably programmed to recognize the choice made by the user.
By way of non-limiting example, dental procedures may be broadly divided into prosthodontic (restorative) and orthodontic procedures, and then further subdivided into specific forms of these procedures. Additionally, dental procedures may include identification and treatment of gum disease, sleep apnea, and intraoral conditions. The term prosthodontic procedure refers, inter alia, to any procedure involving the oral cavity and directed to the design, manufacture or installation of a dental prosthesis at a dental site within the oral cavity, or a real or virtual model thereof, or directed to the design and preparation of the dental site to receive such a prosthesis. A prosthesis may include any restoration such as crowns, veneers, inlays, onlays, and bridges, for example, and any other artificial partial or complete denture. The term orthodontic procedure refers, inter alia, to any procedure involving the oral cavity and directed to the design, manufacture or installation of orthodontic elements at a dental site within the oral cavity, or a real or virtual model thereof, or directed to the design and preparation of the dental site to receive such orthodontic elements. These elements may be appliances including but not limited to brackets and wires, retainers, clear aligners, or functional appliances.
A type of scanner to be used may also be input into intraoral scan application 108, typically by a dental practitioner choosing one among a plurality of options. Examples of scanners that may be used include wand scanners, tray scanners, and so on. If the scanner 150 that is being used is not recognizable by intraoral scan application 108, it may nevertheless be possible to input operating parameters of the scanner thereto instead. For example, the optimal spacing between a head of the scanner and scanned surface can be provided, as well as the capture area (and shape thereof) of the dental surface capable of being scanned at this distance. Alternatively, other suitable scanning parameters may be provided.
Intraoral scan application 108 may identify spatial relationships that are suitable for scanning the dental site so that complete and accurate image data may be obtained for the procedure in question. Intraoral scan application 108 may establish an optimal manner for scanning a target area of the dental site.
Intraoral scan application 108 may identify or determine a scanning protocol by relating the type of scanner, resolution thereof, capture area at an optimal spacing between the scanner head and the dental surface to the target area, etc. For a point-and-shoot scanning mode, the scanning protocol may include a series of scanning stations spatially associated with the dental surfaces of the target area. Preferably, overlapping of the images or scans capable of being obtained at adjacent scanning stations is designed into the scanning protocol to enable accurate image registration, so that intraoral images can be registered to provide a composite 3D virtual model. For a continuous scanning mode (video scan), scanning stations may not be identified. Instead, a practitioner may activate the scanner and proceed to move the scanner within the oral cavity to capture a video of the target area from multiple different viewpoints.
In one embodiment, intraoral scan application 108 includes an image registration module 111, a historical template module 112, an area of interest (AOI) identifying module 115, a flagging module 118 and a model generation module 125. Alternatively, the operations of one or more of the image registration module 111, historical template module 112, AOI identifying module 115, flagging module 118 and/or model generation module 125 may be combined into a single module and/or divided into multiple modules.
Image registration module 111 is responsible for registering or stitching together images received during a scan session. The images may include X-rays, 2D intraoral images, 3D intraoral images, 2D models, and/or virtual 3D models corresponding to the patient visit during which the scanning occurs. In one embodiment, performing image registration includes capturing 3D data of various points of a surface in multiple images (views from a camera), and registering the images by computing transformations between the images. The images may then be integrated into a common reference frame by applying appropriate transformations to points of each registered image. In one embodiment, the images may be registered together in real time, during the current scan session.
In one embodiment, as each new image is received, that new image is compared to some or all previous images received during a current scan session. If there is sufficient overlap between the current image and one or more of the previous images, then the current image may be registered with those one or more previous images. Image registration algorithms are carried out to register intraoral images, which essentially involves determination of the transformations which align one image with another image. Image registration may involve identifying multiple points in each image (e.g., point clouds) of an image pair, surface fitting to the points of each image, and using local searches around points to match points of the two adjacent images. For example, image registration module 111 may match points of one image with the closest points interpolated on the surface of the other image, and iteratively minimize the distance between matched points. Image registration module 111 may also find the best match of curvature features at points of one image with curvature features at points interpolated on the surface of the other image, without iteration. Image registration module 111 may also find the best match of spin-image point features at points of one image with spin-image point features at points interpolated on the surface of the other image, without iteration. Other techniques that may be used for image registration include those based on determining point-to-point correspondences using other features and minimization of point-to-surface distances, for example. Other image registration techniques may also be used.
Many image registration algorithms perform the fitting of a surface to the points in adjacent images, which can be done in numerous ways. Parametric surfaces such as Bezier and B-Spline surfaces are most common, although others may be used. A single surface patch may be fit to all points of an image, or alternatively, separate surface patches may be fit to any number of a subset of points of the image. Separate surface patches may be fit to have common boundaries or they may be fit to overlap. Surfaces or surface patches may be fit to interpolate multiple points by using a control-point net having the same number of points as a grid of points being fit, or the surface may approximate the points by using a control-point net which has fewer number of control points than the grid of points being fit. Various matching techniques may also be employed by the image registration algorithms.
In one embodiment, image registration module 111 may determine a point match between images, which may take the form of a two dimensional (2D) curvature array. A local search for a matching point feature in a corresponding surface patch of an adjacent image is carried out by computing features at points sampled in a region surrounding the parametrically similar point. Once corresponding point sets are determined between surface patches of the two images, determination of the transformation between the two sets of corresponding points in two coordinate frames can be solved. Essentially, an image registration algorithm may compute a transformation between two adjacent images that will minimize the distances between points on one surface, and the closest points to them found in the interpolated region on the other image surface used as a reference. Each transformation may include rotations about one to three axes and translations within one to three planes.
In some instances, a received image may not be registerable with any other image from a current scan session. In traditional systems such images would be discarded, and a user would be asked to rescan one or more areas. Embodiments of the present invention enable such images that do not register to other images from a scan session by comparing and registering such images to a historical template that has been generated from scan data collected in a previous scan session.
Historical template module 112 is responsible for determining that historical scans exist for a particular patient and registering images of a current scan session to a historical template based on overlapping data from the images and the historical scan data. The past historical templates may be or include X-rays, 2D intraoral images, 3D intraoral images, 2D models, and/or virtual 3D models of the patient corresponding to past visits of the patient and/or to dental records of the patient. The past historical templates are included in reference data 138 for the patient.
In one embodiment, historical template module 112 may receive an indication from computing device 105 that a current image of an intraoral scan session cannot or should not be registered to any other images of the current session. Registering may be improper when the current image does not adequately overlap any other existing image of the current scan. In this example, historical template module 112 may determine that the images of the current scan are related to historical image data for the same patient. Historical image data may include historical templates for the patient; each template relating to a previous image scan session for the patient. In one example, a patient may have undergone an intraoral scan once a month for the last 12 months, in which case 12 historical templates for the patient would be available. Historical template module 112 may determine that the images of the current scan are associated with the historical templates and map the current images of the current scan session to one of the historical templates (e.g., the most recent template). In another embodiment, a user may select a historical template to which the current scan session images should map.
AOI identifying module 115 is responsible for identifying areas of interest (AOIs) from intraoral scan data (e.g., intraoral images) and/or virtual 3D models generated from intraoral scan data. In one embodiment, areas of interest are determined by comparing images of a current scan session to a historical template to determine any differences. Such areas of interest may include voids (e.g., areas for which scan data is missing), areas of conflict or flawed scan data (e.g., areas for which overlapping surfaces of multiple intraoral images fail to match), areas indicative of foreign objects (e.g., studs, bridges, etc.), areas indicative of tooth wear, areas indicative of tooth decay, areas indicative of receding gums, unclear gum line, unclear patient bite, unclear margin line (e.g., margin line of one or more preparation teeth), and so forth. An identified void may be a void in a surface of an image. Examples of surface conflict include double incisor edge and/or other physiologically unlikely tooth edge, and/or bite line shift. The AOI identifying module 115 may, in identifying an AOI, analyze patient image data 135 including historical templates (e.g., historical 3D image point clouds) and/or one or more virtual 3D models of the patient alone and/or relative to reference data 138. The analysis may involve direct analysis (e.g., pixel-based and/or other point-based analysis), the application of machine learning, and/or the application of image recognition. AOI identifying module 115 may additionally or alternatively analyze patient scan data relative to reference data in the form of dental record data of the patient and/or data of the patient from prior to the patient visit (e.g., historical templates including one or more prior-to-the-visit 3D image point clouds and/or one or more prior-to-the-visit virtual 3D models of the patient).
In an example, AOI identifying module 115 may generate a first historical template of a dental site based on a first scan session of the dental site taken at a first time and later generate a second virtual model of the dental site based on a second scan session of the dental site taken at a second time. The AOI identifying module 115 may then compare the first historical template to the second virtual model to determine a change in the dental site and identify an AOI to represent the change.
Identifying of areas of interest concerning missing and/or flawed scan data may involve the AOI identifying module 115 performing direct analysis, for instance determining one or more pixels or other points to be missing from patient scan data and/or one or more virtual 3D models of the patient. Flagging module 118 is responsible for determining how to present and/or call out the identified areas of interest. Flagging module 118 may provide indications or indicators regarding scan assistance, diagnostic assistance, and/or foreign object recognition assistance. Areas of interest may be determined, and indicators of the areas of interest may be provided, during and/or after an intraoral scan session. Such indications may be provided prior to and/or without construction of an intraoral virtual 3D model. Alternatively, indications may be provided after constructions of an intraoral virtual 3D model of a dental site.
Examples of the flagging module 118 providing indications regarding scan assistance, diagnostic assistance, and/or foreign object recognition assistance will now be discussed. The flagging module 118 may provide the indications during and/or after an intraoral scan session. The indications may be presented (e.g., via a user interface) to a user (e.g., a practitioner) in connection with and/or apart from one or more depictions of teeth and/or gingivae of a patient (e.g., in connection with one or more X-rays, 2D intraoral images, 3D intraoral images, 2D models, and/or virtual 3D models of the patient). Indication presentation in connection with depictions of patient teeth and/or gingivae may involve the indications being placed so as to correlate an indication with the corresponding portion of the teeth and/or gingivae. As an illustration, a diagnostic assistance indication regarding a broken tooth might be placed so as to identify the broken tooth.
The indications may be provided in the form of flags, markings, contours, text, images, and/or sounds (e.g., in the form of speech). Such a contour may be placed (e.g., via contour fitting) so as to follow an extant tooth contour and/or gingival contour. As an illustration, a contour corresponding to a tooth wear diagnostic assistance indication may be placed so as to follow a contour of the worn tooth. Such a contour may be placed (e.g., via contour extrapolation) with respect to a missing tooth contour and/or gingival contour so as to follow a projected path of the missing contour. As an illustration, a contour corresponding to missing tooth scan data may be placed so as to follow the projected path of the tooth portion which is missing, or a contour corresponding to missing gingival scan data may be placed so as to follow the projected path of the gingival portion which is missing.
In presenting indications (e.g., flags), the flagging module 118 may perform one or more operations to pursue proper indication display. For instance, where indications are displayed in connection with one or more depictions of teeth and/or gingivae (e.g., a corresponding virtual 3D model), such operations may act to display a single indication rather than, say, multiple indications for a single AOI. Additionally, processing logic may select a location in 3D space for indication placement.
Where indications are displayed in connection with a 3D teeth and/or gingiva depiction (e.g., in connection with a virtual 3D model), the flagging module 118 may divide the 3D space into cubes (e.g., voxels corresponding to one or more pixels of the 3D space). The flagging module 118 may then consider the voxels relative to the voxels of determined AOIs and tag voxels so as to indicate the indications (e.g., flags) to which they correspond.
Where more than one cube is tagged with respect to a given one of the indications, the flagging module 118 may act such that only one of the tagged voxel receives flag placement. Moreover, the flagging module 118 may choose a particular voxel which it determines will provide for ease of viewing by the user. For instance, such choice of a voxel may take into account the totality of indications to be flagged and may endeavor to avoid crowding a single cube with multiple flags where such can be avoided.
In placing indications (e.g., flags) the flagging module 118 may or may not take into account factors other than seeking to avoid crowding. For instance, the flagging module 118 may take into account available lighting, available angle, available zoom, available axes of rotation, and/or other factors corresponding to user viewing of the teeth and/or gingiva depiction (e.g., the virtual 3D model), and may seek indication (e.g., flag) placement which seeks to optimize user viewing in view of these factors).
The flagging module 118 may key the indications (e.g., via color, symbol, icon, size, text, and/or number). The keying of an indication may serve to convey information about that indication. The conveyed information may include classification of an AOI, a size of an AOI and/or an importance rank of an AOI. Accordingly, different flags or indicators may be used to identify different types of AOIs. For example, pink indicators may be used to indicate gingival recession and white indicators may be used to indicate tooth wear. Flagging module 118 may determine a classification, size and/or importance rank of an AOI, and may then determine a color, symbol, icon, text, etc. for an indicator of that AOI based on the classification, size and/or importance rank.
Turning to keying which conveys AOI classification, indicators may identify classifications assigned to intraoral areas of interest. For examples, AOIs may be classified as voids, changes, conflicts, foreign objects, or other types of AOI. AOIs representing changes in patient dentition may represent tooth decay, receding gums, tooth wear, a broken tooth, gum disease, gum color, moles, lesions, tooth shade, tooth color, an improvement in orthodontic alignment, degradation in orthodontic alignment, and so on. Different criteria may be used for identifying each such class of AOI. For example, a void may be identified by lack of image data, a conflict may be identified by conflicting surfaces in image data, changes may be identified based on differences in image data, and so on.
When a scan session is in progress or when the scan is complete (e.g., all images for a dental site have been captured), model generation module 125 may generate a virtual 3D model of the scanned dental site. During a scan session, model generation module 125 may use the image registration data output by image registration module 111 to generate a three dimensional representation of the dental site. The three dimensional representation may additionally include image registration data output by historical template module 112 (e.g., registration of current images to a historical template). AOI identifying module 115 and/or flagging module 118 may perform operations to identify AOIs and/or to indicate such AOIs in the three dimensional representation.
Model generation module 125 may integrate all images into a single virtual 3D model by applying the appropriate determined transformations to each of the images. In one embodiment, model generation module 125 may determine that a historical template associated with the current scan session includes areas not yet scanned in a current scan session. In such a case, model generation module 125 may use data from the historical template to supplement the data of the current scan. For example, if there are any regions that have not yet been scanned in a current scan session, a data from a historical template may be used to fill in the missing regions. In one embodiment, the data from the historical template is shown using a different visualization than the data from the images received during the current scan session. For example, surfaces associated with images from the current scan session may be shown as solid objects, and surfaces associated with the historical template may be shown as translucent objects a pixelated objects.
FIG. 2A illustrates a flow diagram for a first method 200 of referencing a historical scan during an intraoral scan session, in accordance with embodiments of the present invention. This method may be performed by processing logic that comprises hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (such as instructions run on a processing device), or a combination thereof. In one embodiment, processing logic corresponds to computing device 105 of FIG. 1 (e.g., to a computing device 105 executing an intraoral scan application 108).
At block 202 of method 200, an intraoral scan session of a dental site is started by a dental practitioner. The scan session may be for an intraoral scan of a partial or full mandibular or maxillary arch, or a partial or full scan of both arches. The dental practitioner may move an intraoral scanner to a first intraoral position and generate a first intraoral image. At block 204, processing logic receives a plurality of intraoral images. The intraoral images may be discrete images (e.g., taken from a point-and-shoot mode) or frames of an intraoral video (e.g., taken in a continuous scanning or video mode). The intraoral images may be three dimensional (3D) images having a particular height, width and depth.
At block 206, processing logic determines that one or more of the intraoral images of the current scan session does not register to any already existing images associated with the current scan session. In one embodiment, images of the scan session may not register to existing images when they do not sufficiency overlap each other. Images do not sufficiently overlap each other when there is insufficient overlapping data between the two images to be able to determine that a portion of the first image is of the same object (e.g., region of a dental site) as a portion of the second image. When images do not sufficiently overlap each other, registration may be less accurate or may not be possible. Images may be registered when a sufficient area and/or number of points within each image may be mapped to an adjacent image. In one embodiment, images whose overlapping area and/or number of overlapping points do not meet a predetermined threshold may not be registered.
In one embodiment, when an intraoral image of a current scan session is not registered to any other image of the current scan session, processing logic at block 208 may determine that the images of the current scan session are associated with an existing historical template. The historical template may be a three dimensional virtual model of a dental site as generated from images taken during a previous scan session. The historical template may have been generated after stitching together the images taken during the previous scan session. Alternatively, processing logic may determine that scan data exists from previous scan sessions, but no historical template may have been generated. In such an instance, processing logic may generate a historical template from the previous scan data. In one embodiment, processing logic may search a database to identify historical templates associated with images of the current scan session or with the current patient. Current session images may be compared to previously captured images to determine whether they are associated. In another embodiment, scan sessions are associated by virtue of the fact that they are associated with the same patient in a database. A user may also manually identify stored historical templates for a patient and associate them with the current scan session.
At block 210, processing logic may determine the position of the first image of the current scan session with respect to the historical template and register the image to the template. In one embodiment, the remaining images of the current scan session at a second region of the dental site may be registered to the historical template at a first region of the dental site (see block 212). In one embodiment, images are registered to the historical template using a first set of criteria, whereas images are registered to each other using a second set of criteria. In one embodiment, the first set of criteria is less restrictive than the second set. In one embodiment, images may be registered to a historical template under the less restrictive criteria when they may not be registered to other images under the stricter criteria.
Accordingly, image data from the historical template may be provided to a user with associated images from the current scan session overlaid in position over the historical template. Alternatively, the historical template may not be provided to a user and the images of the current scan session are provided to the user placed on an invisible historical template. Images of the current scan session that are not registered to other images of the current scan session may still be registered to the historical template. Unregistered images of the current scan session, despite not having sufficient overlap to register to current scan session images, may have sufficient overlap with areas of the historical template to allow placement onto the historical template. Determining proper placement of an unregistered image based on the historical template allows unregistered images to be registered to other images of a current scan session, despite insufficient overlap with those images. In one embodiment, processing logic may utilize the same techniques discussed above with respect to registering images to determine placement of an image with respect to a historical template.
In one embodiment, processing logic may generate a three dimensional virtual model of the dental site from the intraoral images. Processing logic may determine that a portion of the intraoral images represents soft tissue (e.g., gums) based at least in part on a difference between a previous depiction of the dental site as represented in the historical template and a current depiction of the dental site as represented in the intraoral images. The portion of the intraoral images determined to represent the soft tissue may be removed from the three dimensional virtual model of the dental site.
In another embodiment, processing logic of method 200 may identify a restorative area of the dental site represented in the plurality of intraoral images. The restorative area may include a stump (e.g., a preparation) of a tooth. Processing logic may also determine a baseline area that represents a surface of the tooth before the tooth was ground into the stump based on the historical template and provide the baseline area to be displayed in a graphical user interface.
FIG. 2B illustrates a flow diagram for a second method 220 of referencing a historical scan during an intraoral scan session, in accordance with embodiments of the present invention. This method may be performed by processing logic that comprises hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (such as instructions run on a processing device), or a combination thereof. The first method 200 and second method 220 may be performed together or separately in embodiments. In one embodiment, processing logic corresponds to computing device 105 of FIG. 1 (e.g., to a computing device 105 executing an intraoral scan application 108).
At block 222 of method 220, processing logic receives a plurality of intraoral images during a intraoral scan session. The intraoral images may be discrete images (e.g., taken from a point-and-shoot mode) or frames of an intraoral video (e.g., taken in a continuous scanning or video mode). The intraoral images may be three dimensional (3D) images having a particular height, width and depth.
At block 224, processing logic determines that one or more of the intraoral images of the current scan session does not register to any already existing images associated with the current scan session. In one embodiment, images of the scan session may not register to existing images when they do not sufficiency overlap each other. Images do not sufficiently overlap each other when there is insufficient overlapping data between the two images to be able to determine that a portion of the first image is of the same object (e.g., region of a dental site) as a portion of the second image. When images do not sufficiently overlap each other, registration may be less accurate or may not be possible. Images may be registered when a sufficient area and/or number of points within each image may be mapped to an adjacent image. In one embodiment, images whose overlapping area and/or number of overlapping points do not meet a predetermined threshold may not be registered.
In one embodiment, when an intraoral image of a current scan session is not registered to any other image of the current scan session, processing logic at block 226 may determine that the images of the current scan session are associated with an existing historical template. The historical template may be a three dimensional virtual model of a dental site as generated from images taken during a previous scan session. The historical template may have been generated after stitching together the images taken during the previous scan session. Alternatively, processing logic may determine that scan data exists from previous scan sessions, but no historical template may have been generated. In such an instance, processing logic may generate a historical template from the previous scan data. In one embodiment, processing logic may search a database to identify historical templates associated with images of the current scan session or with the current patient. Current session images may be compared to previously captured images to determine whether they are associated. In another embodiment, scan sessions are associated by virtue of the fact that they are associated with the same patient in a database. A user may also manually identify stored historical templates for a patient and associate them with the current scan session.
At block 228, processing logic may determine the position of the first image of the current scan session with respect to the historical template and register the image to the template. In one embodiment, the remaining images of the current scan session at a second region of the dental site may be registered to the historical template at a first region of the dental site.
At block 230, processing logic may display the registered first intraoral image at the first region on the historical template. In another embodiment, processing logic may display additional images on a second region of the historical template. Accordingly, image data from the historical template may be provided to a user with associated images from the current scan session overlaid in position over the historical template. Alternatively, the historical template may not be provided to a user and the images of the current scan session are provided to the user placed on an invisible historical template. Images of the current scan session that are not registered to other images of the current scan session may still be registered to the historical template. Unregistered images of the current scan session, despite not having sufficient overlap to register to current scan session images, may have sufficient overlap with areas of the historical template to allow placement onto the historical template. Determining proper placement of an unregistered image based on the historical template allows unregistered images to be registered to other images of a current scan session, despite insufficient overlap with those images. In one embodiment, processing logic may utilize the same techniques discussed above with respect to registering images to determine placement of an image with respect to a historical template.
In another embodiment, processing logic of method 220 may identify a restorative area of the dental site represented in the plurality of intraoral images. The restorative area may include a stump (e.g., a preparation) of a tooth. Processing logic may also determine a baseline area that represents a surface of the tooth before the tooth was ground into the stump based on the historical template and provide the baseline area to be displayed in a graphical user interface.
FIG. 3A illustrates a flow diagram for a method 300 of generating a surface of a dental site during an intraoral scan session, in accordance with embodiments of the present invention. This method may be performed by processing logic that comprises hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (such as instructions run on a processing device), or a combination thereof. In one embodiment, processing logic corresponds to computing device 105 of FIG. 1 (e.g., to a computing device 105 executing an intraoral scan application 108). In one embodiment, method 300 continues from block 212 of method 200.
At block 302 of method 300, processing logic may determine a first surface represented by a first intraoral image and a second surface represented by a second intraoral image. The first surface may be associated with a first region of a dental site and the second surface may be associated with a second region of the dental site that is separated from the first surface of the dental site. At block 303, processing logic may determine a third surface and/or additional surfaces at additional regions of the dental site based on a historical template. The third region may separate the first region and the second region.
At block 304, processing logic generates a pictorial representation of the dental site that includes the first surface and the second surface. In one embodiment, the pictorial representation may include a transformed representation of the first intraoral image (e.g., that has been rotated, translated and/or scaled) at the first region (determined in block 206 of FIG. 2A) and transformed representations of the remaining intraoral images in the second region (see block 212). The transformed first intraoral image may define a first surface of the dental site and the transformed remaining intraoral images may define a second surface of the dental site. The pictorial representation may optionally also include the third surface and/or any additional surfaces generated from the historical template. At block 306, processing logic may display the pictorial representation in a graphical user interface (GUI) or may output the pictorial representation for display on a GUI.
One or more additional images including the third region may be received by processing logic and registered to the first intraoral image of the first region and one or more of the remaining intraoral images of the second region. The one or more additional images may then be used to redraw the third surface based on current patient data.
FIG. 3B illustrates a flow diagram for a method 310 of determining a change to a dental site between a current intraoral scan session and a past intraoral scan session, in accordance with embodiments of the present invention. This method may be performed by processing logic that comprises hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (such as instructions run on a processing device), or a combination thereof. In one embodiment, processing logic corresponds to computing device 105 of FIG. 1 (e.g., to a computing device 105 executing an intraoral scan application 108).
At block 315 of method 310, processing logic may generate a virtual 3D model of a dental site from images taken during a current scan session. The virtual 3D model may be generated by registering and stitching together the images. At block 320, processing logic compares the virtual 3D model to a historical template generated from images associated with a previous scan of the dental site at an earlier time. The historical template may be another virtual 3D model of the dental site that was generated from images taken during a previous scan session. For example, a dental practitioner may perform scans of a patient's dentition on an annual basis. Processing logic may compare the current state of the patient's dental site to the state of the patient's dental site that is a year old. In additional embodiments, multiple historical templates may be available for a patient, which may be based on images taken at different time periods. A dental practitioner may select which historical template to compare the virtual 3D model to.
At block 325, processing logic determines a difference between a previous depiction of the dental site as represented it the historical template and a current depiction of the dental site as represented in the virtual 3D model generated from the plurality of intraoral images of the current scan.
To compare a virtual 3D model of a current scan session to a historical template and determine differences there between, processing logic determines an alignment between the virtual 3D model (or images used to generate the virtual 3D model) and the historical template based on geometric features shared by the virtual 3D model and the historical template. This comparison may identify areas of interest such as tooth wear, cavities, and so forth.
At block 330, processing logic provides graphical representation of the difference between the current depiction of the dental site and the previous depiction of the dental site as represented in the historical template. In one embodiment, processing logic compares the virtual 3D model to multiple historical templates, and may additionally compare the historical templates to one another. Processing logic may then generate a moving image (e.g., a video) showing a change to the dental site over time. For example, a time lapse video may be generated that shows gradual wearing of a patient's teeth, yellowing of the teeth, receding gum lines, and so on.
The differences between the current and previous depiction of the dental site may be identified as one type of area of interest. In other embodiments, other types of areas of interest may be provided. For example, the provided area of interest may include a region of the dental site that is represented in the historical template but not represented in the intraoral images of the current scan. In one embodiment, processing logic interpolates a shape for the intraoral area of interest based on geometric features surrounding the intraoral area of interest and/or based on geometric features of the intraoral area of interest (if such features exist). For example, if the intraoral area of interest is a void, then the regions around the void may be used to interpolate a surface shape of the void. The shape of the intraoral area of interest may be displayed in manner to contrast the intraoral area of interest from surrounding imagery. For example, teeth may be shown in white, while the intraoral area of interest may be shown in red, black, blue, green, or another color. Additionally or alternatively, an indicator such as a flag may be used as an indication of the intraoral area of interest. The indicator may be remote from the intraoral area of interest but include a pointer to the intraoral area of interest. The intraoral area of interest may be hidden or occluded in many views of the dental site. However, the indicator may be visible in all or many such views. For example, the indicator may be visible in all views of the scanned dental site unless the indicator is disabled. The provided indications of the intraoral areas of interest may be displayed while the intraoral scan session is ongoing.
FIG. 4 illustrates a flow diagram for another method 400 of referencing a historical scan during an intraoral scan session, in accordance with embodiments of the present invention. This method may be performed by processing logic that comprises hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (such as instructions run on a processing device), or a combination thereof. In one embodiment, processing logic corresponds to computing device 105 of FIG. 1 (e.g., to a computing device 105 executing an intraoral scan application 108).
At block 402 of method 400, processing logic receives one or more images of a dental site. Processing logic may perform image registration to stitch these images together if multiple images are received. As the images are stitched together, a virtual 3D model of the dental site that includes the stitched together images may be generated and displayed.
At block 404, processing logic receives a new image of the dental site. At block 406, processing logic compares the new image to some or all of the previous images. At block 408, processing logic determines whether the new image can be registered to any of the previous images. The new image can be registered to a previous image if first registration criteria are satisfied. The first registration criteria may include a threshold number of curves, edges, point and/or other features of the new image that match curves, edges, points and/or other features of a previous image. A match for a feature may be determined when the feature of the new image deviates from the feature of the previous image by less than a deviation threshold, such as 10%. Collectively, the number of matching features and the degree of match may be referred to herein as an amount of overlap. Accordingly, images may be registerable if the amount of overlap between those two images satisfies an overlap threshold. If there are an insufficient number of features in the new image that match features of any previous image (amount of overlap is less than an overlap threshold), then registration cannot be performed and the method proceeds to block 412. If there are a sufficient number of features in the new image that match features of at least one previous image (amount of overlap meets or exceeds the overlap threshold), then the method continues to block 410.
At block 410, processing logic registers the new image to one or more of the previous images using the first registration criteria. Processing logic may then update the 3D model of the dental site to include the new image stitched to the previous images and display the updated 3D model.
At block 412, processing logic registers the new image to a historical template using second image registration criteria. At block 412 processing logic may also register the previous images to the historical template if the previous images have not already been registered to the historical template. The historical template may be a virtual 3D model of the patient's dental site that was generated based on images taken at a previous time. The second registration criteria may be less restrictive than the first registration criteria. For example, a patient's dental site may change over time. Teeth may become slightly worn and change shape, teeth may move slightly, and so forth. Accordingly, images of the same tooth may not match perfectly. To account for these changes over time and still enable a current image to register to images and/or a virtual model generated from images taken at a previous time, the registration criteria are relaxed. The relaxed registration criteria may include a reduction in the threshold number of features that are specified to register two images and/or an increase in the amount of deviation that is permitted for recognition of a match between two features.
Registration of the new image and the previous images to the historical template enables processing logic to determine the regions of the dental site represented by the new image and the previous images. Accordingly, once the new image and the previous images have been registered, the relative positions of the new image and the previous images can be determined. Processing logic may update the 3D model of the dental site based on the registration to the historical template. As a result, the 3D model may show a first region of the dental site represented by the previous images that have been stitched together and may show a second region of the dental site represented by the new image. There may be a small or large separation between the first region and the second region. For regions of the dental site for which image data has not yet been collected in the current scan session, processing logic may show an outline of these regions based on data from the historical template.
At block 418, processing logic determines whether the current scan session is complete. If the scan session is not complete, the method returns to block 404 and another new image of the dental site is received. As new images are received, these new images may be registered to the previous images and/or to the historical template. Eventually, enough images should be generated such that all images can register to one or more other images from the current scan session. If at block 420 the scan is complete, the method continues to block 420.
At block 420, processing logic may refine the image registration between the images from the scan session. The 3D model that is generated during the scan session may be a rough model. Once the scan session is complete, a more rigorous image registration process may be performed to generate a much more accurate 3D virtual model of the dental site. The more rigorous image registration process may use more processing resources and take more time to generate than the 3D model generated during the scan session.
FIG. 5 illustrates a pictorial representation (e.g., a 3D virtual model 500) of a dental site that includes image data from a current scan and image data from a historical template, in accordance with embodiments of the present invention. In one embodiment, the 3D virtual model 500 includes surfaces 502-506 based on images of a current scan that are overlaid onto corresponding positions of an associated historical template 508. The surfaces 502-506 may be shown as solid surfaces.
Each of the surfaces 502-506 may be based on a single image from the current scan or based on multiple images from the current scan that have been stitched together. The images associated with surface 502 could not be appropriately registered to any of the images associated with surface 505 or surface 506 due to insufficient overlap of area between the images. Similarly, the images associated with surface 505 could not be appropriately registered to any of the images associated with surface 506 due to insufficient overlap of area between the images. However, the images associated with surfaces 502-506 may still be registered to the historical template 508. This may enable all of the images from the current scan session to be used rather than discarded. Additionally, by registering the images to the historical template, the 3D virtual model 500 may be generated that shows regions of a patient's dental site for which additional images should be generated.
In one embodiment, the regions of the dental site for which current scan data is not yet available are shown based on data from the historical template. A surface of such regions may be represented by a 3D point cloud, or may be represented as an outline or translucent surface rather than a solid surface. In one embodiment, the virtual 3D model that includes data from the historical template 508 is provided to a user with surfaces 502-506 overlaid on the historical template 508. Surfaces 502-506 may be displayed in color or filled-in (in non-3D point cloud form) so as to differentiate surfaces 502-506 based on the actual scanned images from the surfaces based on underlying historical template 508. Surfaces 502-506 based on intraoral images may be displayed on the historical template 508 in real-time, as they are being recorded. In another embodiment, data from historical template 508 is not provided for display, but surfaces 502-506 based on intraoral images are provided for display in their proper place, based on registration to the historical template 508. Surfaces 502-506 may be displayed in their proper places in real-time, without providing the 3D point cloud historical template 508 for display.
In one embodiment, restorative models may be based on historical template 508. For example, if a dental practitioner is designing a new crown for an affected area of a patient's intraoral cavity, historical templates, such as template 508, may be provided to the practitioner from a time in which the affected area was healthy. An area of the historical template may be selected as a restorative model for the crown. The model may be further adjusted, as necessary, to compete the design of the crown.
In other embodiments, when a user finalizes a scan without scanning an area that was previously scanned and present in a historical template 508, the missing area may be filled in by the corresponding data from the historical template 508. This may be represented by an area changing from display of the historical template's 3D point cloud to display of the same area being filled in (e.g., similar to surfaces 502-506). In one embodiment, the missing area may be filled in based on trends associated with more than one historical template. For example, if three historical templates exist, and each template shows a 1 mm shift in the teeth associated with an area missing from the current scan between scans, the area may be provided with an additional 1 mm shift in the teeth associated with the area.
FIG. 6 illustrates a diagrammatic representation of a machine in the example form of a computing device 600 within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed. In alternative embodiments, the machine may be connected (e.g., networked) to other machines in a Local Area Network (LAN), an intranet, an extranet, or the Internet. The machine may operate in the capacity of a server or a client machine in a client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine may be a personal computer (PC), a tablet computer, a set-top box (STB), a Personal Digital Assistant (PDA), a cellular telephone, a web appliance, a server, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term "machine" shall also be taken to include any collection of machines (e.g., computers) that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.
The example computing device 600 includes a processing device 602, a main memory 604 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM), etc.), a static memory 606 (e.g., flash memory, static random access memory (SRAM), etc.), and a secondary memory (e.g., a data storage device 628), which communicate with each other via a bus 608.
Processing device 602 represents one or more general-purpose processors such as a microprocessor, central processing unit, or the like. More particularly, the processing device 602 may be a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, processor implementing other instruction sets, or processors implementing a combination of instruction sets. Processing device 602 may also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like. Processing device 602 is configured to execute the processing logic (instructions 626) for performing operations and steps discussed herein.
The computing device 600 may further include a network interface device 622 for communicating with a network 664. The computing device 600 also may include a video display unit 610 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 612 (e.g., a keyboard), a cursor control device 614 (e.g., a mouse), and a signal generation device 620 (e.g., a speaker).
The data storage device 628 may include a machine-readable storage medium (or more specifically a non-transitory computer-readable storage medium) 624 on which is stored one or more sets of instructions 626 embodying any one or more of the methodologies or functions described herein. A non-transitory storage medium refers to a storage medium other than a carrier wave. The instructions 626 may also reside, completely or at least partially, within the main memory 604 and/or within the processing device 602 during execution thereof by the computer device 600, the main memory 604 and the processing device 602 also constituting computer-readable storage media.
The computer-readable storage medium 624 may also be used to store an intraoral scan application 650, which may correspond to the similarly named component of FIG. 1. The computer readable storage medium 624 may also store a software library containing methods for an intraoral scan application 650. While the computer-readable storage medium 624 is shown in an example embodiment to be a single medium, the term "computer-readable storage medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The term "computer-readable storage medium" shall also be taken to include any medium other than a carrier wave that is capable of storing or encoding a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present invention. The term "computer-readable storage medium" shall accordingly be taken to include, but not be limited to, solid-state memories, and optical and magnetic media.
It is to be understood that the above description is intended to be illustrative, and not restrictive. Many other embodiments will be apparent upon reading and understanding the above description. Although embodiments of the present invention have been described with reference to specific example embodiments, it will be recognized that the invention is not limited to the embodiments described, but can be practiced with modification and alteration within the spirit and scope of the appended claims. Accordingly, the specification and drawings are to be regarded in an illustrative sense rather than a restrictive sense. The scope of the invention should, therefore, be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.
displaying, by the processing device, relative positions and orientations of the first surface and the second surface, wherein the historical template is not displayed.
registering the one or more remaining intraoral images of the plurality of intraoral images to the historical template at the second region of the dental site.
performing registration between the one or more remaining intraoral images using second registration criteria, wherein the second registration criteria are more restrictive than the first registration criteria.
registering, the by processing device, the first intraoral image of the plurality of intraoral images to one or more remaining intraoral images of the plurality of intraoral images in a three-dimensional space according to the historical template, wherein the first intraoral image is associated with a first region of the dental site that does not overlap with a second region of the dental site associated with the one or more remaining intraoral images.
generating, by the processing device during the intraoral scan session, a representation of the dental site comprising the first intraoral image at the first region and the one or more remaining intraoral images at the second region, wherein the representation is to be displayed on a graphical user interface (GUI).
including the third surface in the representation.
providing an indication of the intraoral area of interest.
generating a graphical representation of the difference between the previous depiction of the dental site and the current depiction of the dental site.
9. The method of claim 4, wherein determining that the first intraoral image does not satisfy the registration criterion with respect to any other intraoral image of the plurality of intraoral images comprises determining that the first intraoral image does not overlap with the one or more remaining intraoral images.
providing the baseline area to be displayed in a graphical user interface.
registering the additional intraoral image to at least one intraoral image of the remaining intraoral images.
removing the portion of the plurality of intraoral images that represents the soft tissue from the three dimensional virtual model of the dental site.
registering the one or more remaining intraoral images to the historical template at the second region of the dental site using the first registration criteria.
register the first intraoral image to one or more remaining intraoral images of the plurality of intraoral images in a three-dimensional space according to the historical template, wherein the first intraoral image is associated with a first region of the dental site that does not overlap with a second region of the dental site associated with the one or more remaining intraoral images.
generate, during the intraoral scan session, an image of the dental site comprising the first intraoral image at the first region and the remaining intraoral images at the second region, wherein the image is to be displayed on a graphical user interface (GUI).
include the third surface in the representation.
provide an indication of the intraoral area of interest.
generate a graphical representation of the difference between the previous depiction of the dental site and the current depiction of the dental site.
provide the baseline area to be displayed in a graphical user interface.
register the first intraoral image of the plurality of intraoral images to one or more remaining intraoral images of the plurality of intraoral images in a three-dimensional space according to the historical template, wherein the first intraoral image is associated with a first region of the dental site that does not overlap with a second region of the dental site associated with the one or more remaining intraoral images.
register the one or more remaining intraoral images to the historical template at the second region of the dental site using the first registration criteria.
23. The non-transitory machine-readable storage medium of claim 22, wherein the processing device is further to perform registration between the one or more remaining intraoral images using second registration criteria, wherein the second registration criteria are more restrictive than the first registration criteria.
remove the portion of the plurality of intraoral images that represent s the soft tissue from the three dimensional virtual model of the dental site.
International Search Report and Written Opinion of International Application No. PCT/US2017/035820 dated Aug. 17, 2017.


text:
Q: Problem in obtaining a cursor! I have a database(SQLite) which contains a tabel with GPS data:longitude and latitude!I'm sure that the database exists and contains the data because I've succed to obtain a cursor all over the data and I've displayed it on the screen.
Now,my app should be a little much bigger than writing in a database and display the data.I should send the data through a socket to another app.
For that I've created another thread which connects to the other app and opens the database and reads the data and sends it to the other app.
public class screen1 extends Activity {

      public void onCreate(Bundle savedInstanceState) {

    Thread cThread=new Thread(new ClientThread());

    cThread.start();

     db=new DBAdapter(this);

   }

And here is my ClientThread class;
public class ClientThread implements Runnable{

       private PrintWriter out=null;
        public void run()

        {

            try
            {
                InetAddress serverAddr=InetAddress.getByName(serverIpAddress);

             socket=new Socket(serverAddr,ClientPort);

    PrintWriter out = new PrintWriter(new BufferedWriter(new OutputStreamWriter(socket.getOutputStream())),true);

            Log.d(" ","Clientul is connected");

            }
            catch(UnknownHostException e){
                 System.err.println("Don't know about host");
            }
            catch (IOException e) {
                System.err.println("Couldn't get I/O for the connection to host");
                     }

            try{
             db.createDatabase();
                     db.openDataBase();
                Cursor  c=db.getAllData();
                  Log.d(" ","before the cursor");
             if(c.moveToFirst())
                {

                  do{
                      Log.d(" ","after the cursor");

                      longitude=c.getString(1);

                     latitude=c.getString(2);

                     Log.d(longitude,latitude);
                  }while(c.moveToNext());

                     }

             socket.close();
                 }
            catch (IOException e) {
               System.err.println("IOException:  " + e);
           }
        }
}

Now the problem I'm facing is this one-I'm trying to get my longitude and latitude displayed but my program never reaches to get into this loop:
if(c.moveToFirst())
                {

                  do{
                      Log.d(" ","after the cursor");

                      longitude=c.getString(1);

                     latitude=c.getString(2);

                     Log.d(longitude,latitude);
                  }while(c.moveToNext());

                     }

How I've tested that? ....=rI've used messages  like:  Log.d(" ","before the cursor")-before that loop and also    Log.d(" ","after the cursor").
Now the first one is always displayed but the second one never is.
Does someone knows what is the problem?Because if I try to get a cursor over the data outside this thread everything works just fine-the data  is displayed....and I'm doing the same thing!
Another thing is that if I use the statement   if(!c.moveToFirst()) the program enters the loop and the second message is displayed buut I get this error in the logcat:
04-19 07:51:31.450: ERROR/AndroidRuntime(611): android.database.CursorIndexOutOfBoundsException: Index 0 requested, with a size of 0
04-19 07:51:31.450: ERROR/AndroidRuntime(611):     at android.database.AbstractCursor.checkPosition(AbstractCursor.java:580)
04-19 07:51:31.450: ERROR/AndroidRuntime(611):     at android.database.AbstractWindowedCursor.checkPosition(AbstractWindowedCursor.java:214)
04-19 07:51:31.450: ERROR/AndroidRuntime(611):     at android.database.AbstractWindowedCursor.getString(AbstractWindowedCursor.java:41)
04-19 07:51:31.450: ERROR/AndroidRuntime(611):     at test.android.screen1$ClientThread.run(screen1.java:96)
04-19 07:51:31.450: ERROR/AndroidRuntime(611):     at java.lang.Thread.run(Thread.java:1096)

android.database.CursorIndexOutOfBoundsException: Index 0 requested, with a size of 0

..What is that?...........So at that cursor is my definitly uniquee problem,but what?
Thank u in advance for your usefull reply!!!I'm here for further details!!!
UPDATE:
public class DBAdapter extends SQLiteOpenHelper {
public static final String DATABASE_PATH = "data/data/test.android/database";
public static final String DATABASE_NAME = "GPSdata";

public static final String DATABASE_TABLE_1= "route";
public static final String DATABASE_TABLE_2= "car1_data";

public static final String KEY_ROWID_1= "_id";
public static final String KEY_NAMEOFCAR="name_of_car";

public static final String KEY_ROWID_2 = "_id";
public static final String KEY_LONGITUDE= "longitude";
public static final String KEY_LATITUDE = "latitude";
public static final String KEY_ROUTE= "route";

public SQLiteDatabase db;

private final Context myContext;

public DBAdapter(Context context) {
    super(context, DATABASE_NAME, null, 1);
    this.myContext = context;

}

@Override
public void onCreate(SQLiteDatabase db) {
    // check if exists and copy database from resource
    createDB();

}

@Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
    Log.w("SqlHelper", "Upgrading database from version " + oldVersion
            + " to " + newVersion + ", which will destroy all old data");
    onCreate(db);
}

public void createDatabase() {
    createDB();
}

private void createDB() {

    boolean dbExist = DBExists();

    if (!dbExist) {

        copyDBFromResource();

    }

}

private boolean DBExists() {

    SQLiteDatabase db = null;

    try {
        String databasePath = DATABASE_PATH + DATABASE_NAME;
        db = SQLiteDatabase.openDatabase(databasePath, null,
                SQLiteDatabase.OPEN_READWRITE);
        db.setLocale(Locale.getDefault());
        db.setLockingEnabled(true);
        db.setVersion(1);

    } catch (SQLiteException e) {

        Log.e("SqlHelper", "database not found");

    }

    if (db != null) {

        db.close();

    }

    return db != null ? true : false;
}

private void copyDBFromResource() {

    InputStream inputStream = null;
    OutputStream outStream = null;
    String dbFilePath = DATABASE_PATH + DATABASE_NAME;

    try {

        inputStream = myContext.getAssets().open(DATABASE_NAME);

        outStream = new FileOutputStream(dbFilePath);

        byte[] buffer = new byte[1024];
        int length;
        while ((length = inputStream.read(buffer)) > 0) {
            outStream.write(buffer, 0, length);
    }

        outStream.flush();
        outStream.close();
        inputStream.close();

    } catch (IOException e) {

        throw new Error("Problem copying database from resource file.");

    }

}

public void openDataBase() throws SQLException {

    String myPath = DATABASE_PATH + DATABASE_NAME;
    db = SQLiteDatabase.openDatabase(myPath, null,
            SQLiteDatabase.OPEN_READWRITE);

}

@Override
public synchronized void close() {

    if (db != null)
        db.close();

    super.close();

}

public long insertData1(String name_of_car) {
    ContentValues initialValues = new ContentValues();
    initialValues.put(KEY_NAMEOFCAR, name_of_car);

    return db.insert(DATABASE_TABLE_1, null, initialValues);
}

public long insertData2(int longitude, int latitude, String route) {
    ContentValues initialValues = new ContentValues();
    initialValues.put(KEY_LONGITUDE, longitude);
    initialValues.put(KEY_LATITUDE, latitude);
    initialValues.put(KEY_ROUTE, route);

    return db.insert(DATABASE_TABLE_2, null, initialValues);
}

public Cursor getAllData()
{
    return db.query(DATABASE_TABLE_2, new String[] {KEY_ROWID_2,KEY_LONGITUDE,KEY_LATITUDE},null,null,null,null,null);
}

public void clearSelections() {
    ContentValues values = new ContentValues();
    values.put(" selected", 0);
    this.db.update(DBAdapter.DATABASE_TABLE_2, values, null, null);
}

}

A: If c.moveToFirst() does return false your cursor seems to be empty. Are you sure that your query was correct and the database is opened etc.
Maybe post some of that code.
Best Regards,
Till

redpajama_set_name: RedPajamaStackExchange


The Original Paint Nite at Local Bars (Up to 36% Off). 60- or 90-Minute Ultimate.Search and compare millions of online tee times and golf deals at thousands of golf courses across the United States, Canada, and 148 other countries.
Discover business offers near you and get notifications about what matters to you. Contact Us about special pre-launch pricing to get your business in the loop today.
Kelowna Best Western offers amazing packages for the wine, ski, or golf lover.
The rugged, desert landscape of Kamloops makes it the perfect spot for a golf getaway.Golf deals and specials near Kamloops, BC. introducing instant booking.
Great businesses offer unique experiences and great savings to members every day on GetintheLoop.
With a long season that often extends from mid-March well into.
Connect with our expert support staff who work to set you up for marketing success.
Access great experiences and offers from businesses around Western Canada from any device with our app or website.Book green fees and tee times in REAL TIME and get your confirmations immediately for Sun Rivers Golf Course in Kamloops, British Columbia.If you live in Kamloops, British Columbia and looking for a Golf Resort, MyCityExplorer Kamloops is the ultimate local business directory for golf resorts in Kamloops.
Book green fees and tee times in REAL TIME and get your confirmations immediately for Tobiano in Kamloops, British Columbia.An online booking service for 5 of Kamloops best golf courses and many accommodation.
Explore the Okanagan region while enjoying free parking, free hotel breakfast and.Sun Rivers Golf Resort in Kamloops: details, specifications and reviews.
Now you can squeeze as much Laughlin as you can into your vacation without putting the squeeze on your wallet.Canada Golf Card is the smartest golf savings program to launch in Canada.
Browse offers and use them immediately online, in-store, or by phone.
Situated 1000 feet above Okanagan Lake, The Rise Golf Course captivates players of all abilities.Grow your business by attracting new customers and building loyalty with call-to-action marketing delivered straight to the smartphones of interested local consumers.


Regardless of whether it's for desktops, portable workstations, tablets, or cell phones, you can simply test your page's ease of use. Duplicate and glue the URL of your website page into the crate gave then pick the determination you need to attempt and tap on "Check". Results can be seen on another page. This is particularly helpful when testing for versatile and tablet resolutions.
On the other hand, the determination could be entered physically. At the point when 'in program' checkbox is checked determination relates to the screen determination of the screen and some space is dispensed for the taskbar and additionally web program controls and outskirts. Else, it is the determination of the website page itself.


City fight tends to concentrate on the metropoliss of the developed universe. There is no literature to turn to the underdeveloped states. Since urbanisation and growing is a planetary phenomena, ca n't metropoliss in the development universe besides find ways to turn to these jobs so they can be attractive, advanced, green and inclusive?
Cities are continually being seen as the critical constituents in fight and invention ( Boddy 2003 ) . It is by and large recognised that the twenty-first century is a century of metropoliss and both cultural and commercial spheres are progressively being distinguished by metropoliss as opposed to states ( Ratcliffe 2004 ) . Rapid and sudden urban growing and development has given rise to sculpt forces of alteration – socially, economically, environmentally and technologically. Consequently, metropoliss are at the Centre of these jobs and contentions. They are intricate adaptative systems, capable to considerable alteration and important uncertainness ( Ratcliffe, 2002 ) .They hold the ability to find the quality of life of the people working and shacking in them.
The traditional ways alongside which metropoliss were planned and administered is turning out to be more unsustainable in the face of accelerated alteration and greater elaborateness in the planetary environment.
The issue at interest remains the inquiry of how urban contrivers, practicians and policy shapers can competently make a metropolis that is competitive-green, advanced, attractive and inclusive-where everyone benefits from the advancement of the metropolis and appreciates diverseness amidst the challenges.
Urbanization in developing states has manifested itself in both negative and positive ways. From the negative facet, in developing states, rapid urbanisation has led to a important addition in environmental debasement, urban poorness, slum development, hapless wellness and sanitation, growing of the informal sector, among others. City fight has been studied and analyzed largely from the economic position ; while the cultural, political, legal, environmental and other such facets have non been given due attending. It is hence the purpose of this research to set up all-around practical schemes of easing metropolis fight in the underdeveloped universe amidst the challenges of urbanisation.
To set up and critically measure the factors that determine metropolis fight.
To happen advanced ways of how to work with or transform the bing challenges into avenues that aid metropolis fight.
To set up the metropolis 's possible and how it can be creatively exploited to progress metropolis fight.
To analyze the different metropolis fight theoretical accounts and develop a theoretical account appropriate for Kampala.
What are the factors that determine metropolis fight?
What are the challenges of urbanisation in Kampala and how do they impede metropolis fight?
How can the bing challenges be transformed into avenues that aid metropolis fight?
What potency does the metropolis have and how can it be exploited to heighten its fight?
What metropolis fight theoretical account can be developed for a metropolis like Kampala?
Begg ( 1999 ) designates the importance of understanding the factors that improve the fight of metropoliss both in the national and international spheres in efficaciously acknowledging their present place and anticipating possible positions for metropolis development. The metropolis environment is defined in two levels- the external and internal degrees. Theoretically, the metropolis 's external enviroment is divided into the planetary and national degrees. This theoretical account identifies one joint degree of the external environment of the metropolis based on Masure ( 2003 ) , Budd ( 1998 ) , European Commission ( 2000 ) and others. They suggest that in the recent old ages the procedures of rescaling have led to the feeling among metropoliss in democratic provinces that the growing and direct influence of planetary forces require the change of the construction and contents of elements in the external enviroment. Others like Webster and Muller ( 2000 ) besides reiterate that the impact of planetary forces has resulted into national factors losing significance. Thes factors include trade liberalisaiton, decentralization, among others. This means that the duty of many of import factors lies at the local degree.
Identifying external fators is the get downing point of acknowledging factors that represent the internal and external environments of a metropolis. These greatly influence metropolis fight. Internal factors of metropolis environment have the greatest importance for metropolis activity results and therefore its fight.
Over the last century, there has been accelerated urbanization in developing states, accompanied by economic growing and development. Over 54 % of the African population will be shacking in urban countries by the twelvemonth 2025 ( UNCHS 1995 ) . Despite the fact that these urban countries in Africa generate 55 % of GNP, the growing is attended by a mass of jobs ( Rugunda 2001 ) . As metropoliss continue to turn, they gain influence and are besides influences by forces outside them ( Mukwaya 2004 ) .
A metropolis is defined as a political-administrative territorial entity with a clearly demarcated boundary and aggressively differentiated legal powers vested in governments on either side of the boundary ( Harris 2007 ) .
" Urban fight refers to the ability of an urban part to bring forth and market a set of merchandises ( goods and services ) that represent good value ( non needfully lowest monetary value ) in relation to comparable merchandises of other urban parts. Non-tradeables, e.g. , local services, are portion of the competitiveness equation. An urban economic system that produces goods and services for local people of high value relative to monetary value, supports the export economic system of the metropolis, doing it more competitory, every bit good as straight raising the quality of life and criterion of life for people populating in the urban part " ( Webster and Muller 2000 ) .


Codemasters' tactical Cold War shooter hoping to heat up sales charts as it ships to retailers nationwide.
With gaming retail rapidly approaching its holiday flash point, Codemasters' Xbox tactical shooter Operation Flashpoint: Elite is rapidly approaching gaming retail.
A ported and expanded version of the 2001 PC game Operation Flashpoint: Cold War Crisis, Flashpoint: Elite includes the original game, as well as the expansion prequel campaign Resistance.
The main campaign of Flashpoint: Elite is set immediately after Mikhail Gorbachev is elected to run the Communist Party of the USSR in 1985, with players leading US forces representing the NATO coalition in taking on a renegade faction of the Soviet Army. The 15-mission expansion campaign, Resistance, is a prequel to the main campaign storyline, casting players as a Soviet resistance leader.
The game also features more than 30 authentic weapons and 40 vehicles for players to use, from Black Hawk helicopters to Abrams tanks. Flashpoint: Elite also includes play over Xbox Live with more than 50 missions (including co-op and team-based outings), and a built-in mission editor.
Developer Bohemia Interactive focused on realism with Flashpoint: Elite, going so far as to model real-time lighting and weather changes, as well as accurate constellations in the night sky by which players can navigate. Bohemia worked on the original Operation Flashpoint: Cold War Crisis, in addition to the Resistance expansion pack, and is currently working on the PC shooters Operation Flashpoint 2 and Armed Assault.
Operation Flashpoint: Elite is rated T for Teen and retails for $39.99. For more information, check out GameSpot's previous coverage.


We are excited to feature Lalicious luxury body care products among our new arrivals at Posh Nails. Lalicious is a refreshing collection of lip butter, body butter, hand cream and sugar scrubs that are infused with tantalizing scents and rich, all natural ingredients that will give your skin the royal treatment.
Each product contains an all star line up of components well known for their skin nourishing benefits: cold pressed coconut oil, sweet almond oil, pure cane sugar, honey, lavender, apricot kernel & macadamia nut oil, safflower & soybean oil, shea butter and cucumber. These ingredients and oils are extracted manually so they retain their vitamins, minerals, and other natural healing properties. Lalicious products are also known for their irresistible fragrances which include hints of summery sugar lemon blossom, brown sugar & vanilla, tropical tiare flower, citrus, rose, and coconut.
All products are paraben free, and can be used during and after your shower time for soft, luminous skin.
Just in time for the new season, Essie debuts its new colors! The collection includes: Dress to Kilt, Take it Outside, Fall in Line, The Perfect Cover Up, Style Cartel, and Partner in Crime. The shades include enigmatic jade, cobalt blue, juicy ripe red, punchy fresh taupe, deep mahogany and intriguing gray–perfect for turning your nails into a fashion accessory!
Each of these trendy shades has an exclusive award winning formula that provides flawless coverage and mirror-like shine, along with excellent, chip-resistant durability. All Essie nail polishes are DBP and toluene free and they can be applied to natural as well as artificial nails.
Make an appointment at Posh Nails today and browse our new collection before enjoying Denver's best manicures & pedicures!
Those irritating bumps and ingrown hairs that sometimes result after waxing or shaving can now be a thing of the past! Flawless by Whish is a soothing gel that helps to prevent ingrown hairs, redness, inflammation, and swelling after hair removal services. Flawless contains all natural ingredients such as organic aloe, willow bark extract, papaya and wheat germ which gently smooth your skin leaving it soft and refreshed.
Capri Blue candles are just as beautiful as they are intoxicatingly fragrant. This premier line of candles is hand-crafted and hand-poured using artisan methods of long ago to deliver outstanding quality. The scents are absolutely amazing as Capri Blue uses a unique super-refined, food grade soy wax formulation that absorbs more fragrance oil than any other natural candle wax in the marketplace. This blend allows the candles to burn clean and strong, lasting up to twice as long as normal candles and perfuming any space with the finest of fragrances.